Data flaws will affect attainment targets
THE Scottish Government aims to reduce the gap in attainment between disadvantaged children and those from more affluent homes. By the end of the current parliament, the Government hopes to be able to show that the gap has significantly narrowed and, at the same time, attainment generally has improved. These aspirations are highly laudable, but they require credible attainment statistics and the ability to relate these to appropriate measures of poverty.
While it is relatively straightforward to measure attainment in the later years of secondary education through the qualifications achieved by learners, measuring attainment among younger learners is much more challenging. The Scottish Survey of Literacy and Numeracy (SSLN) provided a generally respected measure at two stages of primary education and one stage in early secondary. It had its limitations. It covered a relatively small, but important, part of the curriculum. Not every learner was tested; SSLN worked on a sample basis. It was not, therefore, possible to say anything about performance at the individual school level. It could, of course, have been expanded to cover more subject areas or to take in a larger sample. However, the Government has chosen instead to discontinue it. We were promised that better data would be provided in its place.
The expectation was that these better data would take the form of the results of the new standardised assessments to be taken by all pupils in the year groups affected. This kind of testing has disadvantages: it encourages “teaching to the test” and causes curricular areas not tested to be neglected. However, it does generate objective evidence. Unfortunately, hard choices have to be made. Does improving the quality of information justify accepting the associated disadvantages?
The Government has evidently decided that the answer is “no”,
It was not possible to say anything about performance at the individual school level
because although the tests have been introduced, the outcomes will not be made publicly available. Instead, they will be used to “inform” judgments made by teachers.
The measure of attainment that is made public is the proportion of learners achieving the Curriculum for Excellence level relevant to their age, based on teachers’ judgments. However, research has shown that teachers can be much too optimistic about their own pupils’ attainment. Teachers may try to fulfil the policy desire to reduce the attainment gap and raise overall attainment, thereby assessing their students more positively than they should.The data published thus far is of little value.
It would, however, allow newspapers to compile league tables, thus frustrating one of the government’s objectives. Schools would be offered an incentive to improve their positions by giving pupils the benefit of the doubt.
Of course, the attainment figures have to be matched against some measure of disadvantage. The measure the Government has chosen is the Scottish Index of Multiple Deprivation (SIMD). However, this is an area-based measure which is incapable of providing the individuallevel data required. SIMD cannot distinguish between deprived and non-deprived learners living in the same areas. It is significant that, when the Government distributed its Pupil Equity Fund to schools, its measure of relative poverty was not SIMD but entitlement to free school meals. There are other options the government could consider including the development of a Unique Learner Number as recommended by the Commission on Widening Access.
It is clear that valid and reliable assessments of attainment can only be made if there are appropriate measurement tools and data. These are currently not in place with the result that the Government cannot legitimately say whether its attainment policies are working.
Agenda is a column for outside contributors. Contact: agenda@theherald.co.uk