The offensive Katie Hopkins is a symbol of diversity in the narrowing field of press comment

The iconoclastic Hopkins generates huge traffic for her employer, the Daily Mail. Her sharp tabloid prose is sometimes witty but frequently repulsive. It’s hard to imagine many serious newspaper readers could stomach much of her relentless depiction of Britain as a nation “broken” by the “one big fat lie” of multiculturalism and her deliberately cataclysmic claims that London is at war with “the patriots of the rest of England”. When writing for the Sun she referred to migrants as “cockroaches” and “feral humans”.

But the fact remains that the underlying fears exploited by Hopkins – and other tabloid columnists who the liberal media like to see as contrarians, such as Rod Liddle, Jeremy Clarkson and Richard Littlejohn – are genuinely felt by a significant portion of the population (in England at least). Serious papers might consider that they need to find columnists who can articulate those concerns without the hate speech that Hopkins and others readily resort to.

'The things people think, but don't say'

For 10 years, Stephen Fleming has closely monitored the comment pages of the UK national press and condensed it into a daily digest that Editorial Intelligence distributes to its subscriber base. He believes that, in spite of the charged political times in which we live, the comment sections have become narrower and more predictable.

“The point about these so-called contrarian voices being much more mainstream than many would like to think they are is absolutely right. I am sure [the papers] would be looking for more people to write these sorts of things if they could find them, the things that people think but don’t necessarily say,” he says.

“Immigration is a biggie. It’s not very fashionable to say ‘I’m not fond of immigration’ but then that is a concern for [many] people. Those views are articulated by people like Katie Hopkins who seems – whether we like her or not – to be very effective at what she does. I think that there has been a bit of a failure [by the press] over the past few years on that. I would say there is now a lack of variety in comment.”

Fleming, a former journalist on the Times and Daily Mail, speaks as a self-confessed member of the liberal metropolitan elite. But he also lives outside of London. “I live in a place called the countryside, where you go in the pub and it’s a very different community to the [media] breakfasts and drinks parties where it is mostly very liberal people with open minds. My experience of the rest of the country isn’t matched by that and perhaps we are in a bit of a bubble.”

Editorial Intelligence, which was founded in 2005 by Julia Hobsbawm (also an honorary visiting professor in networking at London’s Cass Business School), has built a successful business out of analysing and packaging the best of British press and online commentary and stages the annual Comment Awards to celebrate the stars of what it terms “The Commentariat”.

On 15 May, it will host its Comment Conference, at which James Harding, the BBC’s director of news and the former editor of the Times, will deliver a lecture. Editorial Intelligence surprised a few people at last year’s conference by giving a place on the stage to Hopkins where, a week before the EU referendum, she spoke on the subject “Contrarian Comment: Offensive or Provocative?” After the vote, her position seems less marginal.

“It seems that perhaps we were living in an elite bubble and that has been pricked by democracy – we liked democracy when it suited us and now we don’t like it anymore,” reflects Fleming, an avowed Remainer. He recalls that she received a “quite frosty” reception but was unperturbed. “Fair play to her, she was in a room full of liberal types and she held her own and was in fact quite charming.”

Despite this, he stops well short of suggesting any quality paper should give her a column. “I always think ‘Would I want my children to read this and know that there are people who think this way?’ – and the answer is always ‘No’.”

And while he argues that papers urgently need a greater variety of voices, he acknowledges that the wider readerships of serious titles might be hostile to populist comment. “It’s legitimising that thought process,” he says. “All of us think bad things, but we don’t necessarily articulate and act on them, our morality comes into play. In articulating these thoughts for people [populist columnists] make readers think this ‘intelligent person’ thinks it, so it’s a legitimate position for me to have.”

Even the Sun (which Fleming says is, along with the Daily Mirror, doing less comment as it focuses more on online-friendly entertainment content) recently suspended its columnist Kelvin MacKenzie for crossing the line with some repugnant comments on the people of Liverpool and the Everton footballer Ross Barkley.

Serious standards

Serious newspapers need to “maintain standards” in the post-truth era, says Fleming, and by publishing populist writers “they are not really separating themselves from Joe Public who tweets or blogs or puts something on Facebook”. Somehow, they need to find voices from beyond the metropolitan bubble that are decent, as well as readable.

He argues that the Financial Times has set the standard in terms of breaking out of the confines of its traditional subject matter to provide a breadth of commentary on issues important to modern society.

The Daily Mail might argue that it has found no problem in giving a platform to the frustrated voice of middle England, but Fleming says that it too has narrowed in its outlook. “The Mail used to commission people like Yasmin Alibhai Brown. I don’t see any of that now, those different voices,” he says. “I don't think they will be able to dominate that [right-wing] ground forever without diversifying a bit. Eventually people will get fed up with that agenda-led approach.”

Some would argue that this apparent consistency in the Mail’s comment output is exactly what modern readers want in these polarised times.

Fleming also observes that the online-only Independent is “more shouty” than its print predecessor, which he remembers as providing “the best pound-for-pound comment pages in terms of how much they invested”. He thinks the Times also lacks variety, although he praises the Guardian for both its online presentation and the content itself, noting that the paper “has a lot to complain about” in the current political climate.

The past decade has coincided with an explosion of amateur commentary on blogs and social media platforms, rocking the pedestals of the press columnists who once were able to dominate the public’s attention. Fleming argues that the fast-paced modern news cycle puts a premium on the opinions of the best professional commentators. “Opinion has become more important because of the accessibility and ubiquitous nature of news. Everyone knows within minutes that there is a general election and you don’t have to wait for a paper to find that out. But you do rely on a paper for the implications of why are they holding it.”

Fleming, a Londoner, suspects that a London-centric narrowness in press commentary is also contributing to a sense of regional segregation in Britain. It’s another are where more diversity is required.

But, irrespective of whether its being written from the heart of Brexit country or from a study overlooking Hampstead Heath, the finest commentary performs the same purpose, Fleming believes. “The best comment pieces, when you read them, make you think ‘If I was a bit cleverer than I am, then I would have thought of that’,” he says. “I agree with them, they have done my thinking for me and I can tap into that and look a bit brainier.”

Ian Burrell's column, The News Business, is published on The Drum each Thursday. Follow Ian on Twitter @iburrell