Why India’s university ranking system is flawed

In its current format, the Centre’s NIRF reinforces the misplaced belief that you get the education you pay for

As the new admission season for college entrants continues, finding the ‘best’ university has become as important as deciding on the ‘best’ course. The recent ministry of human resource development (MHRD) effort at ranking Indian colleges and universities attempts to answer this but in the process has raised more questions. Is the current National Institutional Ranking Framework (NIRF) exercise going to encourage academia to generate research and teaching?

Interestingly, this ranking exercise is being initiated precisely when spending on education by the government has been falling. Compared to 2013-14, when education got a paltry 4.57% of the budget expenditure, there has been a decline to 3.65% in 2016-17. This makes the ranking exercise even more intriguing, as raising research and teaching quality involves increased expenses. Take, for example, Britain’s well-known Research Excellence Framework of 2014 (REF). This officially cost close to 246 million pounds and was carried out over five years. The REF-ranked journals, established publication protocols and set up a regime for research funding and for rewarding research and teaching output. Despite the criticism against the REF, a considerable amount of time and money was spent to create a transparent evaluation system for judging academic performance and outcomes. The NIRF 2017, in contrast, has lasted less than six months with absolutely no information in the public domain on costs and expertise involved.

The NIRF has collated information under five broad heads: Teaching, Learning and Resources, Research and Professional Practice, Graduation Outcomes, Outreach and Inclusivity, and, Perception.

There are shortcomings in thmethod and execution. Over 70% of the NIRF data is self-reported by the colleges and universities which can potentially be circumspect and needs rigorous verification, but has not been done. Even though summary data has been put in the public domain by the NIRF for the top 100 universities, at least a quarter of their own homepages do not corroborate their data for public viewing. There is thus little chance of verification by transparency. Besides the problem of data reliability, there are methodological issues too that need further scrutiny. The ranking, for example, relies on ‘perception’ in a prominent way. Can such a criterion offer any objective numbers? Or is this just intended to favour universities who advertise and self-promote aggressively. It may encourage universities to game the system by getting higher ranks without necessarily possessing academic quality.

Second, while to its credit the NIRF has relied on Web of Science and Scopus publications, no effort is visible to meaningfully discuss parity of publications in Indian languages. There is an acute need to factor in an innovative weightage system for such disciplines. The NIRF has also been silent on some important aspects involving academic integrity. The framework has no rules to dis-incentivise wrongful academic practices, it has no protocols for tackling ‘predatory journal’ publications or plagiarism, and, the debate on management quotas and academic quality remain untouched. Despite these glaring shortcomings, the University Grant Commission’s (UGC’s) new draft regulations proposes to use the NIRF ranking to allocate its scarce public funds.

In the 2017 NIRF list, two-fifths of the top 100 universities are funded by state governments. These state universities currently enrol almost double the number of students than the private colleges and universities. And yet their funding is a lot less. If no proper weightage is given to this important fact then state universities will always be ranked lower. The linking of the NIRF in its present form and the UGC funds could divert scarce public money to private universities and colleges under a range of pretexts. An impression is created that the NIRF helps students to make informed decisions for enrolment and policy-makers to scientifically grant financial privileges.

The NIRF in its current format reinforces the misplaced belief that you get the education-you-pay-for. It may also lead to further financial disenfranchisement of deserving public universities. The current NIRF framework, therefore, begs an urgent debate.

OVER 70% OF THE NIRF DATA IS SELFREPORTED BY THE COLLEGES AND UNIVERSITIES. THIS CAN POTENTIALLY BE CIRCUMSPECT AND NEEDS RIGOROUS VERIFICATION, BUT THAT HAS NOT BEEN DONE