Guest article: Is data constraining reality?

The following article has been written by David Kernohan, Associate Editor of Wonkhe. 

This post was created exclusively for the Talis Informer, a quarterly newsletter from Talis aimed at those leading and influencing Higher Education libraries. If you’d like to receive the newsletter, please get in touch at info@talis.com.

When I play with data of any sort the first thing I need to do is understand how it is organised.

Generally, data is designed to tell a particular story, and this underlying narrative informs both what is collected and how it is presented.

As I don’t collect my own data (in my role as a troublemaking data journalist) this part of the process – which my colleagues at Wonkhe would probably describe as “mucking about with a spreadsheet” is a great way to learn about the policy impulses that have lead to the expense of data collection and collation. Everything from the unit of analysis to the range of characteristics available for filtering is a deliberate choice with a policy rationale and a policy impact.

In higher education data the unit of analysis is generally the provider – which could be anything from an Old and Ancient University with a Latin name for the boiler house, to a newly minted, business-focused, alternative provider. Though the two are equal in regulatory terms – both will sit on the mighty Office for Students Register, this does not make them sensibly comparable.

The University of A has around 12,000 students, studying across nearly the full range of subject areas – B institute has 60 students, all studying business, accountancy, or management. Students at A live in palatial college accommodation and study full time, students at B live at home and study either part-time or via distance learning. The University of A is one of the richest landowners in the UK, B institute – judging from Companies’ House records – is struggling year-to-year.

And yet – in the Teaching Excellence Framework (TEF), and in Access and Participation data (A&P) we find A and B pitted together as equals. And this is grotesquely unfair on both. If five students out of 60 drop out of B Institute, it has a non-continuation rate of 8.3 per cent. For the University of B the same non-continuation rate would relate to nearly 1,000 students. But both would be treated the same – neither in this case would be seen as a cause for concern.

But wait – imagine that B institute recruits primarily among Black and minority ethnic (BME) students. It has 50 BME students and 10 white students – but when 5 students drop out, they are all BME. So for BME students, B Institute has a non-continuation rate of 10 per cent, for white students a non-continuation rate of 0 per cent. This is now a “continuation gap”, and is thus a problem that could end in regulatory sanctions.

Now, none of this is to say that B Institute has a real problem with BME underattainment. It is, after all, primarily recruiting from that particular demographic group – a provider like the University of A would have around 8 per cent BME students. And in that situation, a continuation gap could be a sign of a problem.

This is a problem caused by an inapplicable unit of comparison. If we compared B Institute with, say, a business school in a larger university – some of these problems will fall away. To be fair, B is very small – even by microprovider standards – but is a useful tool to demonstrate the oddness of comparing providers.

Back when some of us thought Labour had a chance of forming a government, we were offered a vision of a sector that transcended the idea of providers. It was too much for many in the sector, and – indeed – was not attractive to voters either. But it made sense, in terms of a planned skills and post-compulsory sector linked to a wider industrial strategy.

The stumbling block for many was tied up in the idea of providers – autonomous, distinctive, entities that compete in many ways for students. We’ve had this idea for so long it feels natural, but it is – at heart – an artefact of data organisation.

So beware – your arbitrary data clusters are not just a shorthand for describing reality, they are also a way of creating and constraining it. But I would say that, as an Aquarius.

Thank you to David Kernohan for contributing this post. David Kernohan is an Associate Editor of Wonkhe. Until June 2016, he worked at Jisc as a programme manager and senior codesign manager, after being seconded from HEFCE in 2006. He has also worked for the University of Glamorgan (now the University of South Wales). As Associate Editor, David has responsibility for the development and delivery of a variety of editorial content. His key areas of wonkishness include teaching quality enhancement policy, funding policy, sector agency politics and history, research policy, and the use of technology and data in Higher Education.