UK re­searchers say learn­ing gain is poor na­tional com­para­tor

THE (Times Higher Education) - - CONTENTS - Anna.mckie@timeshigh­ere­d­u­ca­tion.com

Mea­sures of stu­dent “learn­ing gain” are too con­text-spe­cific to be used to com­pare uni­ver­sity per­for­mance on a na­tional level, re­searchers have con­cluded.

The now-de­funct Higher Ed­uca- tion Fund­ing Coun­cil for Eng­land spent £4 mil­lion on 13 pi­lots aimed at find­ing ways of track­ing the im­prove­ment in skills and com­pe­ten­cies made by stu­dents dur­ing their de­grees, with one eye on cre­at­ing a new method of rat­ing uni­ver­si­ties’ teach­ing stan­dards.

When the pi­lots in­volv­ing 70 providers were an­nounced in 2015, Jo John­son, who was then the uni­ver­si­ties min­is­ter, said that they would “help as­sess teach­ing qual­ity and ex­cel­lence”, while Madeleine Atkins, He­fce’s chief ex­ec­u­tive, said that the projects had “the po­ten­tial to sup­port mea­sure­ment and in­di­ca­tors at in­sti­tu­tional and even na­tional level”.

How­ever, a con­fer­ence or­gan­ised by one of the pilot groups, led by the Uni­ver­sity of War­wick and in­volv­ing 17 other Rus­sell Group in­sti­tu­tions, heard that its ex­per­i­ments had found that learn­ing gain was not ap­pli­ca­ble na­tion­ally – for ex­am­ple, as a met­ric in the teach­ing ex­cel­lence frame­work.

This comes shortly af­ter He­fce’s suc­ces­sor or­gan­i­sa­tion, the Of­fice for Stu­dents, scrapped a separate pilot that ex­am­ined whether stan­dard­ised tests could be used to mea­sure learn­ing gain in Eng­land. The project strug­gled to re­cruit enough stu­dents to take the ex­ams.

So­nia Ilie, a se­nior re­search fel­low at the Uni­ver­sity of Cam­bridge’s Fac­ulty of Ed­u­ca­tion, told the con­fer­ence con­vened by the Learn­ing and Em­ploy­a­bil­ity Gain Assess­ment Com­mu­nity (Legacy) that early anal­y­sis of a tool be­ing de­vel­oped at Cam­bridge demon­strated that it was able to dis­cern changes in stu­dents’ skills, abil­i­ties and com­pe­tences.

The Cam­bridge ex­er­cise uses a mix of sur­vey and test ques­tions to mea­sure stu­dents’ cog­ni­tive, af­fec­tive, metacog­ni­tive and so­cio-com­mu­nica­tive devel­op­ment.

Dr Ilie said that her team had con­cluded that although the tool could be used ef­fec­tively within an in­sti­tu­tion to pro­vide ev­i­dence for the ped­a­gog­i­cal ef­fec­tive­ness of dif­fer­ent cour­ses, it could not be used for in­sti­tu­tional-level com­par­isons.

“Out­comes show vari­a­tion across time and be­tween stu­dents,” she said. “The gains are not lin­ear, they ebb and flow, and are sub­ject to a whole range of con­tex­tual dif­fer­en­ti­a­tion fac­tors.”

Many of the learn­ing gain tools pre­sented at the con­fer­ence in­volved self-re­port­ing, mak­ing con­text even more im­por­tant, re­searchers ex­plained.

Jan Ver­munt, pro­fes­sor of ed­u­ca­tion at Cam­bridge, said that the point of mea­sur­ing learn­ing gain was “not to com­pare uni­ver­si­ties – I don’t see why we’d want to do that – but to im­prove teach­ing”.

Eluned Jones, di­rec­tor of stu­dent em­ploy­a­bil­ity at the Uni­ver­sity of Birm­ing­ham, added that learn­ing gain was “a use­ful in­stru­ment for in­sti­tu­tions to use in­di­vid­u­ally, but it is not use­ful on a mass scale”. But for­mer uni­ver­si­ties min­is­ter Lord Wil­letts, who also spoke at the con­fer­ence, urged aca­demics to con­sider us­ing their met­rics in a na­tional con­text “how­ever crude they feel the mea­sure may be, be­cause oth­er­wise oth­ers will de­fine it for them”.

Lord Wil­letts pointed out that pol­i­cy­mak­ers were al­ready look­ing at ways to mea­sure learn­ing gain across in­sti­tu­tions and said that it would be bet­ter if re­searchers set the agenda them­selves.

“Aca­demics work­ing on learn­ing gain should pub­lish in­sti­tu­tional re­sults, how­ever im­per­fect, and ex­plain their con­text,” he said. “The UK is cur­rently mea­sur­ing uni­ver­sity im­pact [in the TEF] by what stu­dents are earn­ing six months af­ter grad­u­a­tion, and that is a ter­ri­ble idea,” he added.

Any mea­sures “we use on learn­ing gain just have to be bet­ter than the cur­rent met­rics”, Lord Wil­letts said.

Newspapers in English

Newspapers from UK

© PressReader. All rights reserved.