‘Obsession’ with grade inflation led ministers to ignore expert warnings
Despite being told that the algorithm would penalise disadvantaged pupils, officials ploughed ahead
MINISTERS and the exam regulator Ofqual ignored multiple warnings that its algorithm for grading A-levels and GCSES was heading for catastrophe, according to examiners, advisers, academics, former civil servants and MPS.
UCL was among universities which warned Ofqual as long as four months ago that its assessment model would penalise high-achieving children in disadvantaged schools. Heads from some of Britain’s top schools also submitted evidence to the regulator that there were serious risks of children’s results being downgraded, leading to “inaccurate and unfair” grades.
Even Ofqual’s own expert advisers insisted it was doomed to fail as long as ministers insisted on a system that would prevent grade inflation and ensure results were consistent with previous years. The disclosures follow the admission by schools minister Nick Gibb yesterday that he had been warned by Sir Jon Coles, a former director-general at the Department for Education, that the algorithm would be only 75 per cent accurate, meaning thousands of children would get wrong grades.
Mr Gibb said he had called a meeting with Ofqual to discuss those “very concerns” but said he was “reassured” it would not have that effect.
Despite this, Ofqual ploughed ahead with its algorithm which led to around 40 per cent of last week’s A-level results being downgraded, disproportionately hitting disadvantaged pupils and leading to a public outcry and concerns about GCSES.
Robert Halfon, chair of the Commons education committee, who warned ministers of the impending unfairnesses in the regulator’s plans in a July report, said the MPS’ recommendations to Government and Ofqual were “for the most part ignored”.
In particular, he said Ofqual rejected their demand for the algorithm to be published “immediately” so it could be stress-tested by experts and statisticians to prevent injustices. It was not published until after the results. Even an offer by the Royal Statistical Society to provide two leading univer-submitted sity statisticians to advise on ways of preventing “grade volatility” and effectively assessing pupils’ prior attainment ended up being thwarted after Ofqual demanded they sign non-disclosure agreements.
The regulator also rejected the MPS’ calls for a fairer and less prescriptive appeals system that was not tilted in favour of better-off pupils and parents.
Yesterday Ofqual was due to meet school leaders and teachers’ groups who want answers to their questions on what went wrong, but the regulator cancelled the meeting as it put up what Mr Halfon described as “a Berlin Wall around themselves”.
Schools had believed Ofqual’s algorithm would take account of teachers’ assessments of individual pupils and their prior achievement but, in the end, it was largely based on students being put in rank order and each school’s subject-by-subject historic performance.
David Blow, who sat on the external advisory group on exam grading, which was set up in April to guide Ofqual on the process, said that the process was “doomed to fail” from the moment Gavin Williamson wrote to Sally Collier, Ofqual’s chief regulator, on March 31 giving his instructions for results.
“Mr Williamson said that firstly no child should be disadvantaged and secondly that marks ought to be in line with previous years,” Mr Blow told The Telegraph. “Those are fundamentally incompatible objectives as, to make sure no child is disadvantaged, you need to take an optimistic view of what each could achieve.
“There had been mixed messages; two different questions were posed from the outset, which is the fundamental problem.”
Because of these mixed messages, some schools gave priority to ensuring their pupils were not disadvantaged and gave more lenient grades, while a third of schools gave grades in line with their past performances, he said.
Ofqual modelled a number of algorithms but, when they had received all the teachers’ predicted grades by the end of June, they realised that the marks were too high to use the preferred system and had to use one that would bring down grade inflation, which led to 40 per cent being downgraded.
However, it had been clear that grade inflation would be an issue by June 1, when predicted GCSE grades
75pc The algorithm’s level of accuracy, which schools minister Nick Gibb admitted he was warned about. He said Ofqual had ‘reassured’ him it would not mean thousands of children getting the wrong grades
by 1,900 schools – over half of all state secondaries in England – to the FTT Education Datalab showed massive grade inflation.
A blog by the lab’s statisticians on June 15 revealed that “in every subject we’ve looked at, the average grade proposed for 2020 is higher than the average grade awarded last year. In most subjects, the difference is between 0.3 and 0.6 grades.”
Several of those who work with the regulator told The Telegraph that, as a supposedly independent body, they could and should have done more to push back against the impossible task that had been set by a Secretary of State demanding it prevent grade inflation.
In the responses to the consultation on the grading system, which ran between April 15 and 29, Ofqual were repeatedly warned of the unfairness in the system.
UCL’S Centre for Education Policy and Equalising Opportunities wrote to Ofqual on April 29 to warn that “the use of historic performance data for standardisation could penalise ‘atypical’ students such as high achievers in historically low-performing schools”.
This was four months before Mr Williamson “started noticing such outliers at the weekend”, said Prof Lindsey Macmillan, head of the UCL centre, who noted that there was no solution to the problem as long as Ofqual based its algorithm on school-level data rather than individual pupils.
At the same time, the HMC, which represents Britain’s top independent schools, wrote to Ofqual to warn it that failure to use “objective evidence” of past performance by pupils risked them being graded “downwards”, producing “inaccurate and unfair results in August”. One educational trust told it that using an algorithm to make the grades fit with a school’s historic grades was “a flawed system” which is “disingenuous, dangerous and fundamentally unfair”.
Foreseeing that private schools would come out on top, the chief executive added: “The winners will be those already advantaged by the current system; and the losers will be those already disadvantaged by the current system.”
An exam board even foresaw the backlash, telling the regulator that a “disparity between a centre’s estimations and the final results might also cause an increase in appeals, public criticism, and a lack of public confidence in the summer 2020 results”.
The problems were acknowledged by Ofqual, which admitted that some respondents to its consultation had said its approach would be “unfair to individuals who might have excelled this year and whose grades would be affected by the poor performance of their predecessors”.
One insider said: “Ofqual wanted all the data in from schools on their pupils as quickly as possible, but not enough time was devoted at the beginning to thinking about what the purpose was and what it was going to achieve.
“Because Nick Gibb was so obsessed with preventing grade inflation and consistency year on year, those two things came together to mean that was the only thing they were thinking about. They were not thinking about fairness to individual students and the path this was all going down.”
There was growing alarm by July, with Mr Halfon’s education select committee warning that it was “unconvinced that safeguards – such as additional guidance and practical recommendations – put in place by Ofqual will be sufficient to protect against bias and inaccuracy in calculated grades”.
It culminated in the intervention in early July by former DFE mandarin Sir Jon Coles, who is now chief executive of United Learning, a group of more than 70 independent schools. In a letter to Mr Williamson, he raised particular concerns about using teachers’ predicted grades for small groups of pupils but leaving larger groups reliant on the algorithm. He said this would lead to unfairness in the system.
Ofqual said it had consulted widely and tested different models before selecting the “fairest solution”. This had been developed with input from expert advisers including statistical experts, while it had worked closely with the DFE.