Progress 8 was supposed to be a fair measure of secondary school ‘effectiveness’. New research confirms that it is seriously biased against schools with more disadvantaged pupils.
It is vital to expose this injustice because scoring ‘well below average’ on Progress 8 triggers consequences such as an early Ofsted, forced academisation or transfer to a more ruthless MAT. Poor Progress 8 scores can also lead to families avoiding schools they assume to be of poor quality.
This new report by George Leckie and Harvey Goldstein (University of Bristol) shows that these problems potentially damage schools, particularly those serving poorer neighbourhoods. The researchers look at the impact of the government’s decision not to consider a range of social factors, including disadvantage (measured by free school meal eligibility, but also living in an areas of multiple deprivation).
In particular, Progress 8 fails to recognise that pupils growing up in poverty tend to make less progress – on average, almost half a grade lower in each GCSE subject. For schools with many disadvantaged pupils, this will push down their Progress 8 scores and will often push them below the government -0.5 floor standard (the boundary for being deemed ‘well below average’).
In order to demonstrate the extent of injustice in Progress 8, the Bristol researchers produce an Alternative Progress 8 calculation. This considers poverty, gender, SEN, and ethnicity. This alternative calculation would result in a third of schools changing bands (eg from ‘below average’ to ‘average’). The number of schools targeted for intervention as ‘below the floor standard’ (i.e. ‘well below average’) would fall by a third, from 303 schools to 196. Grammar schools and other schools with few poor or SEN pupils would benefit less than they do under Progress 8.
Their report points out the damage caused by a punitive accountability system, including stress to pupils, deterring teachers away from disadvantaged schools, driving teachers out of the profession, schools gaming by excluding pupils, curriculum narrowing and intensive exam preparation. This is seriously damaging education in all kinds of schools and areas.
The authors are also well known for pointing out the inherent unreliability of all league tables, and the problems this causes when parents use them for choosing between schools.
The report considers the Government’s argument that taking poverty into account would ‘lower expectations’. The authors argue back that punishing schools for teaching disadvantaged pupils is likely to incentivise schools to avoid admitting particular pupil groups (e.g., children with special educational needs), and where they are admitted, to find ways to exclude them before their GCSEs. Consequently, Progress 8 calculations are likely to leave many disadvantaged pupils and their schools feeling as if they have failed. This may dissuade good teachers from working in challenging schools and may induce teachers in those schools to leave.
A video recording of a presentation of this work is available. The slides are also available, as is a working paper with detailed explanation.
This new research is extremely welcome. However it cannot take account of all factors, because the data is simply not there at school level.
- It does not show the demoralisation experienced by young people living in deindustrialised areas with poor employment prospects
- It does not reflect the hidden impact of parents’ own education and the cultural capital which well qualified parents are able to pass on to their children
- It does not show the impact of long-term poverty or extreme hardship such as overcrowding, damp bedrooms, hunger or homelessness.
A year ago schools in the North East came under particular attack. They were supposedly the least effective in England. In a blog post Progress 8 and the North East we exposed the distorted figures behind this attack. The Bristol research confirms that the North East has been unfairly stigmatised as a result of the high levels of poverty in the region: using its alternative measure, the region would emerge as slightly above national average.
In general, schools with many EAL pupils gain through Progress 8 because their use of English develops during the secondary years. There are relatively few in the North East. We also pointed out some hidden factors:
- far fewer graduates in the adult population of the North East than in London and the South East
- demoralisation caused by deindustrialisation and worse employment prospects.
Instead of the current league table and Ofsted culture which pillories schools through public rankings, we need a system which actually helps schools deal with challenging problems.
A decent government would deal with the economic and social issues and remove disadvantage, not just measure it. It would not add to the pain by attacking schools which serve the most disadvantaged areas.
Reclaiming Schools 