Nothing untoward about standardisation

IT IS disappointing that while we have done so well in this year’s National Senior Certificate (NSC) examinations, someone who has little or no experience in the standardisation process could call the entire process into question in a bid to discredit the results.

It must be made clear that the NSC results are not the only results that are quality-assured and standardised by Umalusi. The Independent Examinations Board (IEB) and the Department of Higher Education and Training (DHET), among others, also go through standardisation and are quality-assured by Umalusi.

I will strongly defend the credibility of the team of recognised academics that form the Umalusi board before I take seriously the ill-informed opinions of an opposition MP who holds a qualification in communications, not statistics or education. Umalusi is made up of doctors and professors in the various relevant fields. Gavin Davis is simply not qualified to discredit the intricate process of standardisation conducted by these experts.

The fact that he chose to abuse his parliamentary privilege of having been invited to be part of the audience during the final stages of the standardisation process and chose to engage in public on matters he doesn’t fully understand is beside the point.

For this reason, I am not going to engage his open letter directly, but rather seek to explain the process and reassure those concerned by Davis’s irresponsible ramblings that there is nothing untoward or unusual about standardisation. For Davis’s benefit, Umalusi will provide a workshop for the portfolio committee to help him understand the standardisation process.

Umalusi provides a step-by-step explanation of what standardisation is about and how it is carried out in South Africa and elsewhere in the world, such as England.

Standardisation is the moderation process used to mitigate the effects caused by exam-related factors. This is separate from the pupils’ subject knowledge, abilities and aptitude, which affect their performance.

The standardisation of examination results is necessary to take care of any variation in the standard of the question papers, which may occur despite careful moderation, as well as variations in the standard of marking that may occur from year-to-year. Other variables include undetected errors and pupils’ interpretation of questions.

During the standardisation process (which also involves statistical moderation), qualitative input from external moderators, reports by internal moderators and post-examination analysis reports, and the principles of standardisation are considered.

Standardisation is necessary to achieve comparability and consistency of examination standards over years to mitigate the variables that affect pupil performance from one year to another, for example cognitive demand and the varying difficulty of questions, marking, curriculum changes and interventions. Standardisation aims, in the main, to achieve an equivalent standard of examination over the years, of subjects and of assessment bodies, and to deliver a relatively constant product to the market: universities, colleges and employers.

The results are standardised across the board to ensure a prospective employer or university administrator that an A in one year is the equivalent to an A attained in another year. This is obviously a very crude explanation of a very complex process.

The 2016 NSC results reflect more raw marks than those from 2015. This could be because the examination papers were set more appropriately and the moderation process of setting the papers was improved. The adjustments (decided by the assessment standards committee of Umalusi) consistently follow guiding principles. The committee comprises academics with extensive expertise in statistical moderation, statistics, assessment, curriculum and education.

Although the final stages of the process, namely standardisation, may seem highly statistical, this adjustment is the culmination of a long process of receiving and reflecting on qualitative and quantitative inputs. This starts with the setting of papers, then moderation, writing of exams, marking of exams, verification and only finally the adjustment of mark distributions.

Sometimes they are adjusted upwards and sometimes they are adjusted downwards in individual subjects. The claims of some that it is done in order to manipulate the results doesn’t make any sense. Each learner would have a different subject combination and Umalusi does not work with the overall results of different learners but rather the individual subjects in isolation.

Davis failed to use the opportunity accorded to him to better understand this process and instead chose try to discredit the results of over 800 000 learners, some of whom worked extremely hard to get the results they did. If indeed he intends to be involved in constructive engagement, he should educate himself on matters of education. There are many problems in the system, and we are the first to admit that. We are addressing these challenges, including the teaching of mathematics, providing quality education to those with special needs and infrastructure provision. Davis would find more constructive use of his energy by working with us to find solutions to actual problems than trying to find problems where there are none.

Well done to the class of 2016. Your hard work paid off and you deserve the results you received. You have done us proud. Thank you.