Sur­veys — much ado about noth­ing

China Daily (Hong Kong) - - HK COMMENT - STEPHEN VINES The au­thor is a for­mer news­pa­per ed­i­tor who now runs com­pa­nies in the food sec­tor and moon­lights as a jour­nal­ist, writer and broad­caster.

World rank­ings don’t you just love them! All sorts of out­fits pro­duce th­ese great big lists of com­par­a­tive in­ter­na­tional per­for­mance. But even when the sur­vey’s au­thors say their data is not re­ally suf­fi­cient to make de­fin­i­tive in­ter­na­tional com­par­isons there is an enor­mous flurry of self con­grat­u­la­tion from those at the top of the list and great wails of de­spair from those at the bot­tom.

We saw a clas­sic ex­am­ple of this at the be­gin­ning of the month when the Or­ga­ni­za­tion for Eco­nomic Co­op­er­a­tion and De­vel­op­ment pub­lished the re­sults of its Pro­gram for In­ter­na­tional As­sess­ment — or PISA test for 2012, which com­pares global ed­u­ca­tional sys­tems by test­ing 15-year old stu­dents.

Even cyn­ics seem to have missed the won­der­ful pun in the acro­nym for this sur­vey be­cause the Ital­ian city of Pisa is most fa­mous for its lean­ing tower that looks as if it is about to top­ple at any mo­ment yet re­mains in place.

The same can be said of this sur­vey that was greeted with wild en­thu­si­asm in East Asia where schools scored highly, es­pe­cially in Shang­hai with stu­dents top­ping the math­e­mat­ics, read­ing and sci­ence cat­e­gories. Even here in Hong Kong there were cel­e­bra­tions over mak­ing it to third place in the math­e­mat­ics rank­ings.

It would be churl­ish not to con­grat­u­late Shang­hai’s schools and in Hong Kong, where crit­i­cism of the ed­u­ca­tional sys­tem is rife; the good news has been grasped with some alacrity by the gov­ern­ment

In fact, PISA is one of the bet­ter global sur­veys be­cause it has the in­tel­li­gent ob­jec­tive of as­sess­ing not just what stu­dents know but also what knowl­edge and skills they can use for par­tic­i­pa­tion in daily life. How­ever, the sur­vey’s or­ga­niz­ers have them­selves pointed out that the find­ings are not suf­fi­ciently com­pre­hen­sive to draw sweep­ing con­clu­sions.

This, of course, has not stopped sweep­ing con­clu­sions be­ing drawn. Thus, we have seen much celebration in Asian coun­tries fol­low­ing the sur­vey’s pub­li­ca­tion while in coun­tries such as the United States and those in the Nordic world, where ed­u­ca­tional sys­tems are gen­er­ally highly re­garded, there has been a flurry of self abase­ment.

The prob­lem is that fo­cus­ing on in­ter­na­tional com­par­isons of­ten ob­scures the re­al­ity at ground level. I have a lot of con­tact with ed­u­ca­tors in Hong Kong who reg­u­larly com­plain that their stu­dents lack cu­rios­ity and ini­tia­tive and fo­cus al­most en­tirely on ex­am­i­na­tion re­sults.

Yet Hong Kong does well in in­ter­na­tional sur­veys of ex­am­i­na­tion re­sults. How­ever, look closely at the over­crowded schools in poorer ar­eas and the droopy eyes of bet­ter-off stu­dents sub­ject to far too many hours of ex­tra exam-ori­ented tu­ition and you will see another pic­ture. This one sug­gests the ed­u­ca­tion sys­tem has pro­found prob­lems.

In­ter­na­tional sur­veys don’t pick up on this, but that does not mean in­ter­na­tional com­par­i­son sur­veys have lost their pop­u­lar­ity. One of the sur­veys I like best, for its ris­i­ble qual­ity, is the much pub­li­cized an­nual global sex­ual ac­tiv­ity sur­vey con­ducted by the con­dom maker Durex. This pur­ports to tell us how many times peo­ple in var­i­ous coun­tries have sex.

Hong Kong reg­u­larly ranks very low in this sur­vey, gen­er­at­ing much id­i­otic com­ment about how the pres­sures of lo­cal so­ci­ety mit­i­gate against peo­ple hav­ing sex. Another way of look­ing at this is that when some­one with a clipboard comes around ask­ing how many times you have sex; an hon­est an­swer tends not to be the first re­sponse that comes to mind. In some coun­tries, peo­ple like to boast of their sex­ual prow­ess, in oth­ers, Hong Kong would be an ex­am­ple here, peo­ple are more mod­est and more ret­i­cent in dis­cussing this in­ti­mate sub­ject with strangers.

There­fore, I would con­clude that the value of the Durex sur­vey is around nil, but it is only one in a group of other sur­veys mea­sur­ing global hap­pi­ness, con­tent­ment and good­ness knows what else. It re­ally is non­sense be­cause the sub­ject mat­ter is highly sub­jec­tive and there is no tan­gi­ble way of mea­sur­ing th­ese things.

The same can­not be said of the PISA test but valid ques­tions need to be asked about the test’s meth­ods, about its cul­tural bi­ases and about who par­tic­i­pated in the sur­vey be­cause, gen­er­ally speak­ing, ed­u­ca­tional sur­veys show big dif­fer­ences be­tween the per­for­mances of stu­dents from dif­fer­ent so­cial classes.

Most wor­ry­ing is what hap­pens af­ter th­ese in­ter­na­tional com­par­isons are pub­lished. This can lead to com­pla­cency among pol­icy mak­ers who have no right to be com­pla­cent; alarm, among those who re­ally have no need to be alarmed; and, most dam­ag­ingly, con­cern about things that do not re­ally mat­ter.

How­ever, in­ter­na­tional sur­veys are much loved by the me­dia be­cause they make for good copy. Coun­tries also have an ob­ses­sive ten­dency to pour over in­ter­na­tional per­for­mance com­par­isons.

I re­call be­ing sent as a young reporter to a Bri­tish city that was ranked in some sur­vey as be­ing the most mis­er­able place in Bri­tain. My job, I pre­sumed, was to find lots of mis­er­able peo­ple and make some­thing out of their mis­ery. The city looked rather de­press­ing but some­how the peo­ple re­fused to be as de­pressed as they were sup­posed to be. I in­ter­viewed one older man who could not un­der­stand what all the fuss was about, but pointed to a house across the road and said, ‘mind you Al­bert over there is a mis­er­able bug­ger’. That com­par­i­son made quite a lot of sense.

Stephen Vines

Newspapers in English

Newspapers from China

© PressReader. All rights reserved.