Data science warped under media spotlight: Did YouTube really charge for bot views?
Anyone active in the advertising community will have heard the media hype surrounding a recently published study by European academics that explores the detection of fake views on video platforms. Looking at the press coverage to date, you’d be led to believe that researchers have exposed YouTube as an unethical trader. The findings suggest YouTube filters fraudulent non-human views more aggressively in its organic count than it does for paid placements – implying that advertisers end up paying for views YouTube knows to be fake.
The release of the report - and subsequent media hype - precedes a key planning period for advertisers as they allocate 2016 ad budgets. This report only adds unnecessary confusion. Even WPP CEO Sir Martin Sorrell weighed in on the study, warning Google that they need to improve their practices; otherwise, “marketers will shift their focus back to traditional media such as press and television.”
But is the study as illuminating as it first appears? Deeper analysis of the research quickly shows that results have been blown out of proportion. The study in fact makes a positive case for YouTube: of the platforms tested, YouTube filtered the most fake views generated by the researchers. This was the core finding of the entire paper, yet it was largely overlooked in press coverage.
Furthermore, the sticking point that fuelled the media hype is methodologically unsound. The analysis of paid views was based on a tiny sample size of just 301 paid views, and a budget total of only €5.56. This is nowhere near statistically significant, and it’s possible that this tiny campaign was not filtered effectively due to a lack of relevant data. The researchers admit they struggled to generate a sufficient number of paid views on their test channel, but chose to publish these very thin results regardless.
The researchers also confirmed that the fraudulent views on their test channel were caught, and that the channel owner was banned from uploading and monetising further videos on YouTube. This only proves that Google's fraud countermeasures are highly effective. Overall, it's a technically interesting paper, but the conclusion around monetised views on YouTube is statistically irrelevant given the superficial data set.
The issue of ad fraud is rightfully generating industry-wide dialogue. A 2014 study by the Association of National Advertisers tracked viewership of billions of digital ads over a period of time and found that almost 25 per cent of video ads on the broader web come from fraudulent sources. Google and YouTube by comparison seem to have a very advanced strategy in place to identify fraudulent views; the most recent European study confirms that in its core results.
It’s regrettable that the weakest part of the study – the portion that was methodologically unsound – was put under the brightest media spotlight. Advertisers should consider the limitations of the study and put the findings into perspective, rather than allowing it to muddy the ad landscape further, or stir up unfounded fear around one of the most brand-safe video platforms available as they allocate 2016 budgets. Instead, the industry’s focus should be directed at establishing unified standards on transparency and viewability, and serving viewers precisely targeted, relevant ads, to drive the industry forward.
Andreas Goeldi is chief technology officer at Pixability