How Heineken, Facebook and CreativeX paved new ways into Creative Effectiveness
As one of the Top 100 Best Global Brands, the Heineken company knows its success is a testament to the power of branding
As digital marketing evolves—and more and more opportunities arise to effectively measure impact—understanding what really drives creative effectiveness is key. Heineken wanted to step-change their approach to evaluating and fueling creative effectiveness. Moreover, as one of the world’s most global beer brands, Heineken needed a way to assess their creative effectiveness cross-market, and across the more than 300 brands they own.
The value of creative
At a time when data has helped many brands understand their audience more deeply than ever before, marketers are keen to understand the value of every aspect of their campaigns.
When it comes to creative, however, objectively assessing performance can be tricky—in spite of the fact that creative’s importance is well-established across the industry.
In 2019, Heineken embarked on an ambitious project to face this challenge, and create a step-change in their understanding of what drives creative effectiveness.
Insights from AI
Working closely with Facebook, Heineken partnered with CreativeX, an AI-based creative analytics company, to identify potential elements of creative effectiveness.
“CreativeX uses a combination of vision, audio, and text detection to identify elements present in a creative—e.g, human present, product present, brand mentioned in audio, etc” said Anastasia Leng, CEO of CreativeX.
“Brands use this technology to automate measurement of everything from digital platform best practices to distinctive brand assets and to objectively understand, in real-time, the consistency and impact of the creative decisions they’re making across all markets and language worldwide.”
CreativeX’s technology analysed hundreds of Heineken’s image and video elements, looking at 15 key creative elements including whether branding or people were present. Taking place over two years, the technology examined 190 campaigns across Heineken’s global brands.
To measure the impact, Facebook leveraged brand lift, which relies on Randomised Control Trial based design—gold standard in the industry—to measure incremental impact.
Brand lift helps you quantify the direct impact of a specific campaign on your business results, and the statistical model used in this case found that these creative elements together accounted for over half of the incremental brand value Heineken achieved on Facebook (1).
Within this, the AI technology found that over two-thirds of the value relied on the content and story-telling elements wrapped up in the creatives.
What they learned
Thanks to this three-way collaboration with Facebook and CreativeX, Heineken gained the expertise to connect creative elements to actual business results—and a framework for future success.
Some of the learnings were less surprising than others. Using the right format ratio, including the brand’s name or logo early and designing for mobile with a clear message were as expected.
Other insights were more subtle, however. For instance, Heineken also found the presence of people in video frames as important in driving effectiveness.
“These elements will not provide a full answer or a recipe for a complete creative,” said Ginevra Capece Galeota, Head of Creative Shop for Global Client & Category at Facebook.
“However, these do provide important evidence-based insights into which elements could work in a creative while building a story.”
“Deciding how to tell that story is entirely driven by the irreplaceable role of the creative professionals.” She continued: “we are excited by these findings because these help Heineken creative professionals take away some of the guesswork and give them insight into the effectiveness of some elements without restricting the actual creative process.”
Measuring creative excellence
As a result, they implemented CreativeX’s in-flight measurement tools, which enabled Heineken to begin measuring creative quality globally and in real-time.
With the analysis emphasising the importance of creative, Heineken set the ambitious goal to achieve 100% adoption of brand specific best practices.
What’s more, Heineken brands are now looking to further expand the use of creative data—from contextualizing results of internal campaign sales effectiveness, to informing future creative development.
“Although basic creative elements that we measured account for only part of the full picture, they are foundational to driving effectiveness in the Facebook ecosystem without restricting the creative process and flow,” said Sander Bosch, who—as Global CMI Manager Communication Effectiveness at Heineken—has been leading the project from its inception.
What this means for marketers
While marketers know creative matters, there is no industry consensus on how best to quantify its impact consistently and at scale (2).
Actual contribution by creative assets to the outcome of an advertising campaign has always been subject to a range of estimates.
In this context, AI can be an important enabler, revealing the actual outcome for numerous campaigns on a large scale.
As the partnership between Heineken, CreativeX and Facebook shows, AI-based creative quality solutions help ensure that creative content is fit for success, and that creative teams can focus on what they do best: creativity.
Three key takeaways from Valentina Culatti, Northern Europe Director for Facebook’s Creative Shop
1. A journey of a thousand miles begins with a single step
If you intend to drive short and/or long-term effectiveness with the right creative assets, you cannot solve it all in one go. Start small! Start with one metric, add more metrics and discover the right tools gradually; don't forget to connect creative measurement to business outcomes. Keep improving!
2. Cherish creativity but use tools to scale the right creative elements.
Use AI-based creative quality solutions as a diagnostic tool across large scale campaigns, platforms and geographies. Your brand gains consistency, scale, and actionable recommendations grounded in science.
Once you build a strong foundation to maximise the chance for your creatives to break-through thanks to AI-based creative tools, the full potential of creativity and storytelling will magnify the impact. Iconic campaigns are behind long-lasting effects.
3. Collaboration, collaboration, collaboration
Nothing happens in silo. In order to produce iconic and effective creative assets for your brand, you need a village: not only bringing together internal and external teams across media, creative and measurement, but also innovative tools.
1. A predictive model was built using data from Heineken brand lift studies run from May 2017 to Jan 2020 and feature importance was calculated using the Shapley Additive Explanations (SHAP) method. We restricted our analysis to ad recall and to campaigns that had statistically significant effects. Total number of campaigns analysed was 190.
2. Andrews, Lara; Olojede, Adetunji; Moakler, Robert; Nawathe, Nikhil and Zhou, Mark. “A framework for improving advertising creative using digital measurement”, Published on Facebook Publications, accessed on 11 May 2021, url: https://research.fb.com/publications/a-framework-for-improving-advertising-creative-using-digital-measurement/