The Drum Awards Festival - Official Deadline

-d -h -min -sec


Future of creativity: Image recognition and analysis


By Kevin Joyner, Director of Data Strategy

November 4, 2019 | 5 min read

Digital technologies have transformed advertising. Online, advertising is delivered with infinite variety. It’s personalised. Its campaigns are controlled with programmed rules and automation. Delivery generates an unlimited volume of data, and – in an everwidening virtuous spiral of awesomeness – the campaign delivery data feeds back into the automated controls. The effects of it all, a wonder to behold, are measurable in the highest definition.

Martech Heroes: Jada Balster

Data v images

And yet there’s something left neglected. Even when digital advertising is idealised, there’s still, inevitably, an elephant in the room. It’s the way digital treats the true art and beauty of our industry’s medium. What, we have to ask, has digital ever done for creative? Data can drive the optimisation of campaign targeting and timing, but it hasn’t – yet – done anything truly positive for advertising’s visual form.

In digital agencies, we like to say we do things better than our traditional agency counterparts. We’re in the habit of thinking we’re more highly evolved. Our special power is data. Wherever we apply it (so we say) it makes things better, and we apply it to everything. We apply it to advertising creative, and so we like to say that, because of data, we make creative better.

The reality is that the operation of advertising creative is more complex than we account for. Creative is composed of the subjects of an image, and sometimes the presence of a face, and its emotion; it’s the colours of the ad, and the colours in combination with one another; it’s the copy, the amount of text, its meaning; it’s the composition, the relative position of all the components, and the way they’re all animated, and – sometimes – the way the whole thing is interactive.

The limitations of split-testing

When an art director or a designer produces an advertising creative, or when they judge it, and judge how to vary it or improve it, he or she considers every element of it (some consciously, some instinctively). In contrast, when the digital agency campaign exec judges the same creative, the data they’ll typically use represents none of the visual detail of the creative at all. The normal approach in digital is the split-test. Done correctly, it tells you that one creative variant is working better than another (done incorrectly, it leads to false conclusions). There’s a lot that the split-test will never tell you: it doesn’t tell you what to do next, how to direct the next creative route; it doesn’t tell you where the logo should be, when the call to action should appear, or which selling points to include in which markets. A split-test tells you that a creative variant has performed better, but it doesn’t tell you why.

Just like a little bit of knowledge, a little bit of data can be a dangerous thing. Until now, the digital agency’s approach to optimising creative has been reductive. We prove that a creative variant is the least worst, and we ask for something else to test. Until now, we’ve been unable to say in any detail why one variant is better than another, or how to compose better creative – something that is absolutely crucial given that creative is the most important sales driver in advertising performance, according to Nielsen. The effect of this might sometimes have been to stifle creativity. Split-testing new variations is not the same as having new ideas.

The promise of new technologies

Now for the good news. We are realising the solution. New technology is enabling a truly better way. Previously, it was practically impossible that we would attempt to codify the detailed visual traits of ad creative. Now, machine learning is making this possible.

Using Google Cloud’s Vision API, we are able to analyse ad creatives for key elements – from faces and emotions to calls to action and fonts – to uncover valuable creative insights. By combining machine learning-driven image recognition with performance data from Campaign Manager, we can now get answers to questions such as the best place to put your brand logo within an ad, or which colour schemes work particularly well.

With many ads now being in animated formats such as gif or HTML5, meaning that frameworks such as Pillow or Selenium are necessary to break the ad down into individual frames, this process is no mean feat, but the possibilities it offers advertisers are incredibly exciting. By mapping out every variant of each element of ad creative against performance and occurrence, marketers can make smarter, faster decisions about their creative.

This not only means deprecating campaigns with underperforming visual elements more quickly, but also surfacing new creative strategies that might otherwise be missed. And the resultant ads should deliver better, more personalised experiences that delight customers – the most compelling argument of all.

Kevin Joyner

Kevin Joyner, director of planning and insight, Croud


More from Technology

View all


Industry insights

View all
Add your own content +