Exam grading isn’t getting any tougher
A complex process ensures that all exams are graded fairly and consistently. Alex Scharaschkin explains why exam boards need the help of statisticians, scientists and psychologists on an annual basis
HOW CAN I tell when exam season is over? Well, apart from the fact that I work for an exam board, there’s one easy giveaway: social media. This summer has seen the continuing trend for students to bombard Twitter with their post-exam analysis: good or bad, always passionate and sometimes very witty. We’ve had Charles Darwin monkey memes, calls for grade boundaries to be lowered, regular appearances by Spongebob Squarepants and a whole host of comments I couldn’t possibly repeat here. And now? Tumbleweed…
So what happens next? The exam papers are marked by tens of thousands of examiners who are mostly current teachers. They’ll mark students’ responses (called scripts – AQA has around 7.5 million in total) either on paper or on screen, depending on the subject.
We train them on how to use the mark scheme correctly and how to mark to the right standard. We check samples of scripts as they’re marked to make sure examiners are marking accurately and consistently. We also set hidden questions for examiners marking on screen. This means that, if an examiner isn’t getting it right, we can step in straight away – either providing more training, or giving the scripts to a different examiner to be marked.
But marking is just the first stage in the process. The second stage is “awarding” – where we set the boundaries that will determine the grades that each student will be awarded. So despite all those calls on social media for exam boards to lower the grade boundaries, they haven’t even been set yet. So given that we’re not influenced by suggestions on Twitter, how do we do it?
First of all, here’s why we need to do it. For obvious reasons, exam boards have to set different exam papers each year, and the aim is for these to be no harder or easier than the previous year. In reality, though, there may be slight differences. So if we were to set grade boundaries in advance, students could get a lower grade if their paper was slightly more difficult. Clearly, that wouldn’t be fair, so that’s why we only set grade boundaries after we’ve marked the scripts.
We set grade boundaries for every exam paper, every controlled assessment and coursework component of a subject, every time the assessments are taken and grades are awarded. As well as being fair to students, it also means that standards are maintained year on year.
Setting the grade boundaries
Senior examiners meet and scrutinise students’ exam performances. However, evidence shows that no matter how experienced or talented an examiner is, it’s very hard to allow for small differences in assessment difficulty when comparing scripts from one year to the next.
If we relied solely on examiners’ judgments, there would be a real risk of the standard varying year to year. So in the interest of fairness to all students, a team of statisticians, data modelling specialists, psychologists, scientists and educationalists based at our Centre for Education Research and Practice (CERP) work together to support our expert examiners, and advise AQA on setting grade boundaries. The final step is approval from the standards regulator, Ofqual. The minimum marks for each grade are then confirmed and applied to the marks each student achieved to produce their final grade.
The process takes time. Over July, senior examiners at AQA will have made more than 2,000 grade boundary decisions. We run over 200 awarding meetings, with a separate meeting for every single GCSE and A-level subject. Many meetings last two days or longer, so that examiners can take enough time over decisions to make sure they get them absolutely right.
But you might well ask: how do we maintain standards from one year to the next when things change, like this year’s new 9-1 grades for maths and English GCSES? It’s a perfectly valid question, to which there is a perfectly reassuring answer: exam boards are used to dealing with change.
Before joining AQA, I worked for one of its predecessor boards (AEB – the Associated Examining Board) in the mid-1990s. This was a time of huge change in education – GCSES were still new, and we were preparing for the introduction of modular exams and AS levels.