Could emotion-sensing technology signal the death knell for tracking as we know it?

Proof of the rapid ascent of this fast-emerging research discipline lies in the string of high-profile announcements made in recent months. The year began with Apple’s acquisition of artificial intelligence (AI) startup Emotient which uses AI-based technology to assess emotions by reading facial expressions – a proposition primarily marketed to help brand owners assess viewer reactions to ads.

Soon after, digital marketing agency Jaywing announced a landmark strategic collaboration with Imperial College London’s Data Science Institute to run joint emotion research and analysis to understand and predict audience response.

Meanwhile, MediaCom announced plans to integrate the Realeyes emotion analytics platform into its central content hub. Realeyes uses second by second emotional analytics gathered from detailed analysis of consumers’ faces via webcam to give content an ‘EmotionAll’ score based on attraction, retention, engagement and impact.

“Over the next five to 10 years, better understanding of consumers’ emotional response will be as significant as the recent emergence of online analytics and programmatic,” Jaywing chief executive Martin Boddy predicts.

Opinion is divided however on the nature and the potential impact of this anticipated step change. For Eleanor Thornton-Firkin, head of content and creative development at Ipsos Mori, emotion technology may transform tracking.

“Ipsos Mori currently uses facial coding and EEG [electroencephalography] to understand emotional response to advertising – although facial coding is used more often as it is more scalable, and it can be easily incorporated into an online surveying allowing people to act naturally in their own homes without having to come into a lab-style environment,” she explains.

“We also use implicit reaction time (a measure of unconscious ad or brand impact by quantifying shifts in brand perception on an unconscious level which can be imbedded into online surveys) to understand the underlying impact of emotion.”

Currently, brand owners most interested in this approach are those eager to better understand video content, Thornton-Firkin says. The appeal is obvious as this is where most money is spent and where subtle changes can have significant impact. But as more and more of us wear biometric technology, this could become the new dominant technology if closely linked to what people are watching and when.

“At some stage someone will probably and robustly predict campaign success through emotional response that point may signal the death knell of tracking as we know it,” she adds.

Palle Finderup Diederichsen, head of MediaCom Beyond Advertising EMEA, goes further. Look beyond current uses – which are mainly focused on understanding emotional response to optimise content for different media or markets, and the potential to apply emotion technology to the whole campaign development process, including channel planning, makes it “an extremely powerful proposition”, he believes.

“The future potential to track a whole range of different responses beyond facial tracking – body language or unconscious reactions, for example – is very interesting,” Diederichsen says.

And the step beyond that? Linking emotional response with sales. “It is already proven that emotions correlate with earned media,” he adds. “That, as an indication of intention to buy, is strong, so emotion has a natural sales effect. It’s just a question of having the right technology to prove it.”

Others are less bullish. Following M&C Saatchi’s AI billboard experiment last year, the agency’s chief innovation officer Dave Cox is cautious.

The campaign, for an invented brand of coffee called Bahio, used facial coding and an algorithm to analyse the emotional responses of people passing a digital billboard in order to test and refine different creative components into a finished ad.

Though no-one’s privacy was compromised, feedback suggested that some found it intrusive.

“I like the idea that by researching emotional response using an algorithm to test an ad, no one can insert their own bias,” Cox observes.

“But I fear there might be a tendency to over-simplify. The idea that there are six universal facial expressions underpins many of the facial coding systems now being used, but is this a little blunt?”

Without doubt, more science is needed, says Heather Andrews, UK chief executive of neuroscience-based market researcher Neuro-Insight. “Emotional response is useful, but only insofar as it persuades a person to lodge something in their memory. Lodging or encoding a memory is what makes the difference between transient entertainment and something more meaningful. Only measure emotion, and all you are measuring is enjoyment.”

Emotion and memory – encoding, as that’s what prompts future action, rather than recall – are equally important and moving forward, what will become increasingly important is understanding the relationship between the two.

“As tools and applications evolve we will see brand owners drawing on a broader range of research approaches and metrics,” Andrews predicts. “Already we are seeing greater interest in media context – the differences in emotional response experienced when consuming content on a tablet, TV, laptop or mobile – and this will continue to grow.”

Jaywing’s Boddy agrees. More science is essential to distance the emerging research discipline from gimmicky and PR-generating initiatives, he says.

“Emotion tracking to gauge the intensity of response can be a powerful tool. But work in this field must go deeper before we can emotionally target – targeting creative content to elicit different, distinct reactions according to a particular brand’s specific objectives.”

Asking how important measuring emotion will be in the future is like asking how important research is, adds Boddy: “It won’t change how marketing is done. But it will help us all make better decisions to do what we do more effectively.”

This feature was first published in The Drum's 4 May issue.