On global indices measuring democracy

How do researchers measure democracy? Why is India worried about its ranking on global indices which measure democracy and political participation? What are the strengths and weaknesses of global datasets? Why does India want to make its own democracy ind

The story so far: he VDem Institute’s recent democracy index termed India as “one of the worst autocratisers”. Similar indices have downgraded India’s democratic standing in recent years — India is only ‘partly free’ (Freedom House), is home to a “flawed democracy” (The Economist Intelligence Unit) and is better classified as an “electoral autocracy.” The Indian Government has however refuted these assessments. It now plans to release its own democracy index, which, according to Al Jazeera, will help India “counter recent downgrades in ratings and severe criticisms by international groups”.

TWhy does India care about a democracy index?

From the Varieties of Democracy (VDem) project to Freedom House, there is a consensus that India’s democracy is in peril. Ahead of the election season, these indices and “negative commentary” by think tanks and agencies threaten India’s sovereign ratings and its ranking on the World Bank’s Worldwide Governance Indicators, the Al Jazeera report noted. India has previously denounced all global rating assessments of Indian conditions, from democracy and press freedom to hunger, human development and happiness. Foreign Minister S. Jaishankar in 2021 called the makers of these indices “selfappointed custodians,” who are “not motiveless.” The grouse with democracy assessments is that the methodology is flawed, sample sizes are inadequate, and that these indices favour cultural bias and subjective opinion over objective metrics. India, for instance, ranks between Niger (which is ruled by a military junta) and the Ivory Coast, and is in the same category as Palestine. Any yardstick of democracy — be it fair elections or electoral participation — would suggest

India is “doing as well as any other democracy,” Mr. Jaishankar said.

What data do indices use?

There are many approaches to measuring democracy, some using facts, some judgment and some a mix of both. The four broad types of data that these indices use are — observational data (OD) which is data on observable facts, such as voter turnout rates; ‘inhouse’ coding, where researchers assess countryspecific information using academic material, newspapers, etc.; expert surveys, where selected experts from a country provide a subjective evaluation; and representative surveys, where a selected group of citizens offer judgments. The Office of the United Nations High Commissioner for Human Rights has endorsed the use of observational, objective data over judgmentbased methodology for assessments to make them “more broadly acceptable.” Others, however, find factbased metrics “insufficient”, and expert intervention necessary, to capture onground realities of governance.

Additionally, each index asks and responds to a different question evaluating the health of democracy. While all agree that democracy is a political system in which citizens get to participate in free and fair elections (electoral democracy), indices like VDem’s, Economist Intelligence Unit and the Bertelsmann Transformation Index assess other dimensions as well: is the democracy ‘participatory’, are citizen groups and civil society organisations functional? Are decisions made deliberately, in the best interest of all people, rather than through coercion or minority group interests? Is it egalitarian — are economic and social resources distributed equally? The approach also varies vastly. Some use only two indicators while others have more than 400; the weightage assigned and aggregation model followed also fluctuates across projects. VDem’s researchers code a series of indicators for 12 areas across media, civil society, political parties, and civil liberties and each area is assigned five experts.

What are the limitations of indices?

The first and the most frequently cited criticism is that there is a degree of subjectivity that tugs at the indices’ credibility and precision. Regardless of the scholarly pool and aggregation model used, evaluations are still based on the judgment of researchers and coders, rather than tangible characteristics. VDem’s “egalitarian” indicator, for instance, assesses the equality of social groups in the political arena — an equivocal question in comparison to say, how many political parties are present in the country. However, a project investigated the degree of expert biases in some indices and found them to be limited. Scholar Paul Staniland, who studied VDem’s assessment of India since 1947, concurred, telling the BBC that “there’s not an obvious antirightwing bias.” Another concern is over the scope of countries included in these indices. Only some survey nonindependent and microstates. Smaller countries may thus be overlooked in certain cases. The next criticism is of a perceived ideological discrepancy, partly due to the amorphous definition of democracy itself. Lesotho, which suffered a military coup in 2014, is assigned a higher score than India.

Therefore, there is no singular, perfect democracy index, just like there is no singular definition of democracy. Experts, including Mr. Staniland, agree the indices “capture important bigpicture dynamics and trends” in democracy. They offer ways to benchmark the strengths and weaknesses of regimes, and make different components comparable over time periods and geographies.

The VDem Institute’s recent democracy index termed India as “one of the worst autocratisers”.

While all agree that democracy is a political system in which citizens get to participate in free and fair elections (electoral democracy), indices like VDem’s, Economist Intelligence Unit and the Bertelsmann Transformation Index assess other dimensions as well: is the democracy ‘participatory’, are citizen groups and civil society organisations functional? Are decisions made deliberately, in the best interest of all people, rather than through coercion or minority group interests?

A major criticism is that there is a degree of subjectivity that tugs at the indices’ credibility and precision.