Limits to rankings as a measure of quality
WE have often read reports announcing the annual international university rankings ― usually such news would be accompanied by various officials also giving their take on the matter.
The tone of the importance of the rankings would probably depend on whether the performance has been going up or down. Prospective students also use these ranking tables as a guide for university applications.
Despite the widespread discussion about university rankings, I chanced upon some conversations that made me realise there are many
― including students, prospective students and parents ― who do not have a clear idea as to how universities are ranked. To be fair, much of the methodology that goes into the ranking of universities are not something that may be familiar to the public, perhaps much more so in Malaysia.
Before we get into the ranking methods, we must first know that there are different ranking tables published by different entities that employ different methods or variations of the same method. As a result, the ranking tables will sometimes publish different ranks for the same university. At times, even the list of the top 10 ranked universities are different. We must also be aware that some universities opt to not participate and some others are not ranked together due to factors such as being a postgraduate only university.
There are many factors that are measured in ranking a university and one major aspect is the research activities that are being carried out by a university. This is measured not only in terms of number of publications resulting from research, but also the number of times the published papers are mentioned or cited in other publications. These citations can provide some insight into the perceived importance of the research being reported, although not always necessarily so.
A university’s capacity to do research in the modern era can also be seen as a chicken and egg conundrum. To carry out impactful research requires substantial funding. For a developing nation such as Malaysia, research funding is not as easy to come by. So to start with, our universities are already handicapped in that sense. To break into the top 100 places in the tables requires cutting-edge research that are supported by state-of-the-art facilities. For established universities with billions or even hundreds of millions in its endowment coffers, money is no object when it comes to infrastructure development.
Among the other factors that are also measured include a university’s reputation as viewed by its peers and its international outlook. This particular parameter measures the proportion of international students and staff as well as collaborations with international partners. It is obvious that for many public universities in Malaysia, the primary admissions pool are Malaysian citizens and this would also result in lower scores compared to universities that have a history of providing higher education to students from all over the world.
Despite all this, many of the larger and more established Malaysian universities have seen a steady rise in some of the international rankings over the past few years. Some of the relatively younger private institutions have also made significant progress. This bodes well for higher education in Malaysia in general. But there are limits as to how much we should depend on these rankings as a means of measuring the quality and value of our higher education institutions.
I’m going to do a rather simplistic and definitely non-exhaustive comparison using the Quacquarelli-Symonds rankings of two Malaysian universities ― University of Malaya (UM) and Universiti Kebangsaan Malaysia (UKM) ― against several universities in Ireland and New Zealand. Why UM and UKM? UM because it is the highest ranked Malaysian university in all the international tables at joint 87th place, and UKM (at joint 184th place) because it specifically caters to Malaysia in carrying out much of the teaching and research in Malay and thus may seem less competitive internationally.
Why Ireland and New Zealand? Again my reasons are rather simplistic ― these two countries are English speaking, with many Malaysian students enrolled in their universities and the countries are perhaps more similar to Malaysia than the developed economies such as the US, UK, Japan or Germany.
The ranks for some of the Irish and New Zealand universities are: The University College Dublin (joint 193rd), The National University of Ireland Galway (joint 260th), University College Cork (joint 338th), University of Otago (joint 175th) and University of Canterbury (joint 231st). Due to their fame, you may have heard of some if not all of these universities. You might have also been surprised, like I was, that they are ranked lower than UM and UKM. To be fair, there are Irish and New Zealand universities that rank higher than UM and UKM — Trinity College Dublin and The University of Auckland are two examples.
Does this mean that these lower ranked Irish and New Zealand institutions are not good or respectable universities? I am sure you would agree that it does not mean that at all. When the Times Higher Education rankings are used, UKM will instead be in the 601800th while the University of Canterbury is ranked at 301-350th. This simply shows that there are various parameters that can be changed or tweaked to change the rank of a university.
University rankings are an important benchmark for us to improve our systems. But it should not be the only means of measuring the quality and standards of a university. Furthermore, universities should not change or gear their practices and operations to be wholly subservient to these ranking methods. Some of the companies publishing these rankings make a large profit out of charging consultation or providing services to help universities improve on their rankings.
Universities must be clear about their mandate and their initial purpose of being ― their raison d’etre. Many universities may have been established to serve specific needs ― either a gap in teaching and/or research not available at other institutions, to serve a local population, or both. In many ways, it is the raison d’etre that can also be the strength of a university and perhaps by revisiting such roots, universities may again find their identity that may have been lost in the race to climb the ranking tables.
UM can trace its origins to the need to train medical personnel for colonial Malaya, particularly Singapore and Penang. Universiti Putra Malaysia can trace its beginnings to an agriculture college. Universiti Sains Malaysia was established to address the need for a university in Penang, hence its original name of Universiti Pulau Pinang. UKM was borne of a nationalist movement to establish a university that would uphold Malay as a language of knowledge with seed funding contributed by the young, old, rich and poor in part of a nationwide drive to demonstrate support for the idea of a national university.
Maintaining some semblance of a university’s original identity and keeping true to its raison d’etre will only make the university a stronger entity. Undoubtedly, how the raison d’etre is interpreted must also evolve over time and must not remain rigid. The general improvement achieved can, in turn, result in a rise up the rankings. It should not be other way round of rising in the rankings to find a university’s identity and purpose. This, of course, requires a fine balance that is perhaps easier said than implemented. Our universities can perhaps revisit those ideals and frame it into a global context for maximum international impact.
Universities must be clear about their mandate and their initial purpose of being — their raison d’etre.
The writer is a bioinformatician and molecular biologist heading the Centre for Frontier Sciences, Faculty of Science and Technology and a Senior Research Fellow at the Institute of Systems Biology, Universiti Kebangsaan Malaysia. Email him at firdaus@mfrlab.org