How the BBC’s Shared Data Unit teaches journalists to find the news 'hiding in plain sight'

We live in a data rich society. Every day, reams of public interest data is released into the public domain: far too much data to be properly scrutinised. The BBC Shared Data Unit team was created to produce original and creative reports from public interest data that had been hiding in plain sight. The team is unique in its mission to train other journalists from other outlets during secondments at the BBC with the skills to allow them to do more and more of the same themselves in future.

The strategy

The BBC Shared Data Unit uses computer-aided journalism techniques to scrutinise previously-untouched sources of information. In 2019, the SDU demonstrably strengthened local news output across the UK by giving BBC regional journalists the skills to interrogate data and tell stories of local importance to their communities.

In terms of its own portfolio of work, the team’s journalism this year produced around 270 different stories for the media partners with whom it shares its content. That same original journalism was picked up for 14 separate reports in national newspapers and 11 pieces of coverage on flagship BBC TV and radio programmes including the 1pm national news, Today on BBC Radio 4, BBC Radio 5 Live as well as making national headlines for BBC Reporting Scotland and Wales Today. It has informed 62 local radio reports. In BBC online output alone, its reports in 2019 have drawn nearly 3.4m readers.

We have revealed community resolution orders – informal punishments which do not appear on criminal records - were still being used by police against suspects of violent crimes, despite guidance restricting the orders’ use to low-level offences.

We analysed official figures to reveal the number of people dying while on probation in England and Wales had risen by almost a third in three years amid accusations of "institutional indifference" towards offenders released from custody.

Against a background of a shortage of social housing, Scotland and Wales having brought the Right to Buy policy to an end, and while Northern Ireland consulted on the future of its equivalent scheme and an extension was trialled in England, we produced the largest UK-wide analysis of the impact of this policy.

We used requests under the Freedom of Information Act to report one in two people who appealed in court against a decision to deny them disability benefits was successful between 2013 and 2018. Our research had immediate impact. It was raised at First Minister's Questions to Nicola Sturgeon in Scotland on the day of publication and it was put to the Secretary of State at the Department for Work and Pensions (DWP) Thérèse Coffey on the campaign trail on the same day.

The results

The SDU’s aim is not limited to content creation however; it’s about passing on crucial data journalism skills to the wider industry by training journalists during three-month secondments with our team. The SDU is unique and mould-breaking in that mission. We set out to encourage best practice in working with and reporting on data, and to give regional journalists the skills to interrogate data and tell stories of local importance to their communities. This is where the SDU has had its greatest, invaluable success in 2019.

In March, JPI Media launched its first dedicated data journalism unit led by new news editor Claire Wilde, who was one of the first reporters to go through a secondment with the SDU in winter 2017. Her team of 11 other journalists working on investigations included another four other former SDU secondees in Paul Lynch, Aimee Stanton, Chris McCall and Anna Khoo. Newsquest also launched its own dedicated data investigations unit in June staffed entirely by three former SDU secondees in Joanna Morris, Bev Holder and Vicky Gayle.

On her secondment with us, Claire, now news editor (data and investigations) for JPIMedia, said: “Being a complete novice at spreadsheets isn't a problem at all, all you need is a willingness to learn and you'll do well. It took my journalism skills to the next level and was really good fun at the same time.”

Paul said: “Put simply, the BBC provides excellent training during the three months while you work on your data projects - skills you will take back to your newsrooms and use regularly.”

Aimee said: "I was pretty much a beginner when I first started but we've covered everything from coding and visualisation to advanced excel techniques and R programming language, so I'm leaving the secondment with a really well rounded understanding of data journalism and how I can use the skills back at JPIMedia.”

Joanna, now working for Newsquest's Data Investigations Team, said: “After coming into this with very little knowledge of data journalism, I’m going away with a whole new skill set that I know will be of use in any newsroom I find myself in. The training was varied, interesting and a definite challenge – but it does eventually click into place, even the concepts that seemed most baffling at the beginning!”

Vicky said: "It reignited my sense of purpose as a journalist. The secondment served as a step towards more investigative journalism and now back at work, I'm in a position to explore stories I wasn't able to before.”

The project was a winner at The Drum Online Media Awards 2020. To find out more about The Drum Awards, including which awards are currently open for entries, click here.