Pilot for online exams earns fail
OIA documents reveal problems for schools trialling NCEA papers
Plans to move school exams online by
2020 hit problems after students were locked out of a pilot assessment labelled a “disaster” by one school.
Documents obtained by the Weekend Herald reveal problems with last year’s media studies exam, including an exasperated teacher threatening to go to media and an urgent briefing of Education Minister Chris Hipkins.
The New Zealand Qualifications Authority says the “minor” technical issues resulted from human error, and it is on track to meet the ambitious goal of providing digital assessment for all subjects by 2020, minus a few that are incompatible, such as art.
The handful of pilot exams at the end of last year was a milestone in that effort. If their school opted in, students could choose to sit the digital pilot exams instead of the paperbased version. Results counted.
Things went wrong from the start of the NCEA Level 2 Media Studies pilot. A number of students were presented with multiple log-ins to different subjects, and were unable to open the media studies assessment.
Documents released under the Official Information Act include feedback from one school that, “we were run off our feet solving problems over the whole exam. It had us thinking about pulling out altogether”.
“When good people with proven track records and a well managed computer system have difficulties, you know the system is calibrated too tightly for national success,” the school stated.
“Our supervisors froze when something went wrong and they then picked up the manual to figure out what to do”.
NZQA says the issue arose for students assigned to multiple digital assessments. It was resolved within 10 minutes of the exam starting. Students couldn’t access the other exams as they did not have the codes.
In November, NZQA told new Education Minister Chris Hipkins’ office there was a possibility the issue could reach media, despite it being “a very minor issue”. A quarter of students didn’t log on, but it wasn’t yet clear why: “It could simply be that they got cold feet”.
Later feedback from participating schools outlined other issues including a slow computer that meant words took about 30 seconds to appear, frozen screens, lost internet connection, and copy-and-paste issues.
Last month NZQA revealed another glitch related to the digital exams, after some students were mistakenly given fail grades. The Level 1 and 2 exams were offered in Classical Studies, English and Media Studies.
NZQA’s deputy chief executive of digital assessment transformation, Andrea Gray, said the login issue was down to human error, and was part of the learning process around digital assessments.
NZQA was still on track to meet the
2020 goal, and was convinced of the benefits of online exams.
“These issues arise, and we are
very disappointed when they arise, but they are not show stoppers. And students are telling us, ‘Keep going NZQA’.”
Comparable education systems overseas were also moving assessment online, with Singapore and the Netherlands particularly advanced. Gray said digital assessment was needed because students were increasingly learning on devices.
“And once you get into digital assessment you have more opportunity to personalise it. For example, you could offer an examination at a different time of year from the paper-based examinations”.
More pilots would be run this year, with Level 3 exams being offered for the first time, but only to students who had previously completed digital assessments.
Michael Williams, president of the Secondary Principals’ Association and principal of Pakuranga College, said the technical problems were part of the reason why trial and pilot exams were run.
“We can’t have schools being some archaic blast from a couple centuries ago with no relation to the real world. The challenges for schools are around
equity — how do we provide equitable access to the technology, both for learning and assessment,” he said.
Hipkins said he had been assured that, apart from some minor technical issues, the pilot went well.
“These pilots are opt-in and support is on hand in the event of technical issues. NZQA has processes in place to ensure no student is disadvantaged.
“There is no reason to think the 2020 goal cannot be reached. The purpose of the digital trials and pilots is to learn about what works well, and where improvements need to be made. NZQA is working with schools to ensure this happens.”
We can’t have schools being some archaic blast from a couple centuries ago with no relation to the real world. Michael Williams, Secondary Principals’ Association