LEAGUE tables that rank schools based on just one year’s performance are deeply misleading, according to a model that compares the quality of a school’s teaching with its standing in the tables.
Academic leagues based on one year’s results are “essentially meaningless for the vast majority of schools”, says mathematician Rebecca Hoyle of the University of Surrey. Yet tables of this type are published in England, the US and Australia, while other countries, including Italy, are considering doing the same.
Hoyle and James Robinson from the University of Warwick used a simple mathematical model to simulate the results of pupils from 10 schools over 15 years. In the model, the quality of the school determined 10 per cent of each child’s exam results, with the remainder divided between prior achievement (60 per cent) and family background (30 per cent). These weightings were taken from previous studies of real schools.
Advertisement
The researchers will report in Proceedings of the Royal Society B that the position of a school in the league does not necessarily reflect its quality of education. Even when all the virtual schools were identical, some established themselves at the top of the table, while others quickly sank to the bottom.
Parents react to small random differences between the schools’ test results, and this creates a feedback effect. Better-off families can afford to move house or pay for transport to send their children to a school they perceive to be better. And because family background affects exam results, this can boost the performance of their chosen school. Less popular schools end up with more children from poorer backgrounds, so their results get worse.
“They have shown that parental choice leads to unfairness,” says Harvey Goldstein, a statistician at the Institute of Education in London. The schools become socially segregated, the researchers showed, and each school’s position in the league table ends up reflecting only the demographics of its pupils.
The “value-added tables” being piloted in England attempt to correct for the students’ background. But these are unreliable too, says Hoyle, because once background is taken away, the gap in quality between the best and worst schools is only slightly bigger than the random differences from year to year.
Kathryn Doherty, research director for the US news magazine Education Week, admits that league tables are a “crude measure”. But she points out that not publishing the tables at all wouldn’t necessarily be helpful. “It’s important to hold schools accountable,” she says.
One alternative is to base tables on several years’ data. Hoyle and Robinson found that averaging the schools performances over the previous four years smoothed out the random variations and represented the schools’ quality hierarchy more accurately.