How will we know if Ogilvy’s ‘Maaate’ campaign has made the right impact?

In the long term, it’ll be judged on its impact. But behavior campaigns are, by their nature, difficult to measure. There’s no sales figure that can rise or fall and prove Ogilvy’s thesis, while social media engagement or press attention aren’t strong indicators of a social shift in the real world. To dig into this question, we asked a bunch of strategists, planners and industry experts how they’d measure the effectiveness of a campaign like this – and what else they’d be considering in the process.

How do you solve a problem like... measuring the effectiveness of a behavior change campaign?

Rachel Pashley, group planning head, Wunderman Thompson UK: “Male friendship groups are a powerful way to address violence against women: it’s a difficult feat to tackle the perpetrator head-on because nobody identifies as such – we don’t look in the mirror and see ourselves as a bad person. So, the strategy relies on creating a culture among male peer groups intolerant to ‘toxic masculinity,’ and measurement of our tolerance and acceptance of this should be our focus. Too often, we assume that to solve the big issues in life we need a big solution. Often this is not feasible or realistic. In practice, in the real world, solving big issues relies on lots of small actions taken by all of us: everybody changes or nothing changes.”

Scott Green, growth strategy director, Posterscope: “I understand the attempt to resonate with young men, to not condescend, but seeing the poster I didn’t think it worked. Too many messages. Jocular vernacular doesn’t always land well on a public medium. Misinterpreted, ridiculed, hard to ignore, it grates. In public, the intervention ‘maate’ feels a little lame against the vitriol on display, as does the call-to-action ‘Say maate…’ Measuring behavior should be around what people do rather than what they say, so perhaps the best way to measure its impact is to monitor the shift in online and real-life behavior overall, which will take time. An important ambition and a campaign like this is only part of the solution.”

Shula Sinclair, EMEA & worldwide chief strategy officer, mSix&Partners: “The power of behavioral economics principles is evident in our work where behavior-change objectives are easily measured. Since we can’t measure the conversations the ‘Maate’ campaign is seeking to influence, we agree the strongest short-, medium- and long-term proxy indicators, which should include asking women whether they have seen a change in behavior from men.

“However, the unintended behavioral economics principle that might also be affecting this campaign is confirmation bias – ie mayor Khan’s perceived lack of permission to speak in this space may be impacting how the communication is received. This would be something we would explore through custom qual and quant studies.”

Tom Gray, chief strategy officer, Imagination: “Rather than measuring heaps of data, spend time determining the single measure of success and build your strategy and idea around it.

“You might be tempted to measure campaign metrics – the instances of the behavior (eg how many men intervene in friendship groups) – rather than the ultimate outcome (reduction in violence against women). But you probably want to measure both – partly because of the length of time it can take to recognize the effect on the outcome and partly because you’d want to validate your initial assumptions about the ability of the campaign to drive a certain behavior.”

Dan Hulse, chief strategy officer, St Luke’s: “In behavior change campaigns, perfect is the enemy of good. In a perfect world, communications could talk directly to the perpetrators of violence and intimidation against women or expect bystanders to make a much stronger intervention. But Ogilvy’s research revealed that these wouldn’t work. So they’ve used insight to find the nudge that people will act on in the real world. Research shows that low-level misogyny can develop into violent assaults that, as crime statistics show, are all too measurable. That sad fact is what will make it possible to measure the impact of this campaign.”

Mark Hauser, applied behavioral scientist, The Team: “It’s key we don’t measure such campaigns against unrealistic expectations like ’solving’ a complex issue that likely manifests due to a variety of different factors and contexts. That said, there are ways to gauge whether a campaign like this has been a success. Here are a couple, with evident trade-offs between feasibility and reliability.

“Look for behavioral data sets to see if there is any correlation between exposure to the campaign and actual behavior change. This wouldn’t prove causality but could provide some indication. Alternatively, one can gauge the impact of creative among the target audience following psychologically and contextually relevant exposure to the campaign). This would require skill/expertise to ensure no bias. As with any intervention, it’d also pay to understand how other important stakeholders feel about the campaign, to understand any unintended negative consequences.”

Jamie Parks-Taylor, director of insight analytics, Cream: “Measuring the impact of campaigns on the sentiments or behaviors of large groups is more difficult than looking at traditional media metrics but the framework exists and is widely used – primarily via consumer research, brand tracking and focus groups. Tools like YouGov’s brand index allow daily tracking of nationally representative groups across a range of metrics like ’word of mouth,’ ’impression’ and ’buzz’ etc.

“The same measurement framework could be used to good effect by political campaigns like this from the mayor of London. Especially when combined with other measurement approaches like social listening (with sentiment analysis applied), focus groups and other consumer studies.”

Lori Meaken, executive member, Women in Advertising and Communications Leadership: “With a problem this pervasive, complex and systemic, we can’t task one ad with changing everything, so we must measure the specific behavior (speaking up, challenging your mates’ everyday sexism), among the specific target audience (those men who want to challenge, but don’t know how); and include observed as well as claimed/intended behavior. I’d also explore what else might be contributing positively or negatively to any changes, from Barbie’s cultural currency to the inevitable backlash from the manosphere. Finally, I’d hope over time to see many of us who’ve survived rape and sexual assault saying that although tough and triggering to watch, it’s definitely a positive thing to do.”

Rhian Drummond, senior creative, Folk: “Frustratingly, to change behavior, you have to start small – particularly when it comes to changing deeply engrained attitudes like low-level misogyny. But beyond giving men a one-off catchphrase to call out their friends, the ‘maaate’ campaign lacks a wider, measurable education piece – at least at first glance. David Fanner, the consultant at Ogilvy’s Behavioral Science Practice, mentions spending time observing men in male-dominated spaces – for example, at the gym, barbers or football practice – the outcomes of these conversations and how they’re used to inform a wider public-facing education piece that will help to measure the campaign’s impact.”

Want to add your voice to future conversations? Email me at sam.bradley@thedrum.com.