Looking behind the headlines on the state of our schools
The most recent Pisa assessment of New Zealand’s schools tell us plenty about the state of education - but understanding it all requires deeper digging, writes
Last December’s release of the latest results from the OECD’s 2022 Progress in Student Achievement (Pisa) international assessment of the capabilities of 15-year-olds triggered further debate about the state of our schooling system.
The 2022 results show we remain above the OECD average in reading and science, with scores similar to those of Australia and the United States, and just above the OECD average in mathematics. But since Pisa started in the early 2000s, New Zealand scores have declined significantly across all three areas, with the largest fall occurring in mathematics.
Like a lot of other New Zealand data, the differences in Pisa scores between high and low achievers, students from the most advantaged and the least advantaged socio-economic groups, and students of different ethnicities, have continued to be large. This is a long-standing weakness of our system.
Overall, Pisa 2022 reinforces that there are issues in schooling we need to urgently address, particularly with respect to inequities in our system. Understanding what Pisa is and is not telling us can help us to do that.
For instance, one particular caveat in interpreting the Pisa results is that in 2018 73% of New Zealand students said they had put less effort into the Pisa assessment than they would have, had the
results counted towards NCEA credits. So, Pisa scores may not fully reflect what New Zealand students actually know and can do, and if effort has declined over time, this might help to explain some of the decline in our scores. But it will not be the whole story.
There have been two periods when New Zealand Pisa results have declined the most. The decline, particularly in our mathematics score, between 2018 and 2022 is one of them. The OECD average declined by a similar amount, and the pandemic, with its associated disruption to learning, is likely an important contributor to this.
The other period of significant decline was between 2009 and 2012. One argument advanced for this is that the change in curriculum made during that time added a focus on learning of key competencies at the expense of knowledge acquisition.
Overall, Pisa data does not point to a simple relationship between New Zealand student Pisa performance and the curriculum.
New Zealand student experience of different aspects of mathematics learning was at the OECD average in some areas, and above or below it in others. New Zealand students’ familiarity with mathematical concepts, which in 2022 was significantly below the OECD average, has nevertheless shown some improvement since 2012, while our Pisa scores have stagnated or declined.
Further, Pisa does not just assess what students know, but also how well they can apply their knowledge in situations with which they are not familiar. This aligns well with the curriculum in force from 2010, so perhaps if the curriculum had not changed, our results since then, at least in some aspects of Pisa, may well have been worse.
The curriculum and how teachers are supported to teach it warrants ongoing work, as has been happening under the current and previous governments, but it is important to also consider a broader range of influences known to impact on learning, and about which Pisa offers insight.
In Pisa 50% of New Zealand 15-year-olds reported they were bored in mathematics classes, significantly higher than the OECD average, and only 20% felt inspired or motivated. Just 46% of our students rated their mathematics instruction as 7 or higher on a 10-point scale, compared to 55% of students, on average, across the OECD.
And the data suggests many of our students find that the use of digital devices in school does disrupt their learning, which the new Government has moved to address.
Also, as the Education Review Office has recently highlighted, in 2022 New Zealand students reported a behavioural climate in mathematics classes that was the most negative among OECD countries, although for most negative behaviours reported on, the frequency did not increase from 2012. In addition, students’ sense of belonging at school (which has been declining) and feelings of safety at school are below the OECD average.
Pisa 2022 showed that across the OECD, significant reasons for absence from school, other than sickness, included family responsibilities such as caring for a family member, not feeling safe at school and being bored with school. Factors such as these should be considered as New Zealand looks to address school attendance issues.
So, Pisa helps to build a picture of what is happening for students in our schools. But to achieve actual improvement in engagement and achievement, and particularly in equity of outcomes, we need to dig deeper than the headlines and, informed by evidence, take a comprehensive approach that takes account of all of the influences that are impacting on learning.