Getting the facts right on honesty
Researchers debunk findings of study published in US, restore Chinese citizens’ reputation
Apaper published by professors and students from 14 Chinese universities aimed at restoring the reputation of Chinese citizens for honesty has triggered hot discussion on social media platforms. More than 100 professors and students from the universities, along with a professor from Old Dominion University in the United States, conducted experiments at 500 different locations in China.
Through data analysis, they debunked a paper published in the journal Science, which found that Chinese citizens ranked the lowest in terms of civic honesty.
The Chinese paper, with the title “Unraveling controversies over civic honesty measurement: An extended field replication in China”, was published in Proceedings of the National Academy of Sciences, or PNAS, in the US in July.
It challenged the findings of a study conducted by four behavioral economists, including Alain Cohn from the University of Michigan, in 2019.
Cohn’s study involved a transnational experiment in which foreign research assistants handed in lost wallets to employees at public institutions such as hotels and banks, and measured civic honesty based on the response rate to email notifications about the wallets being found. The paper found that China ranked lowest among 40 countries.
Yang Qian, a professor at Zhejiang University’s School of Public Health, said the conclusions reached by Cohn’s study caused significant doubt within the academic community due to limited criteria and failure to consider cultural differences between countries.
As the first and corresponding author of the paper “Unraveling controversies over civic honesty measurement”, Yang added: “It is thought that Cohn did not conduct extensive pilot experiments. In Western thinking, email is considered the standard method of communication.”
Sun Yacheng, a professor at Tsinghua University’s School of Economics and Management, and Zhou Xinyue, a professor at Zhejiang University’s School of Management, contacted Science to question Cohn’s research methods and conclusions.
However, the editor of the journal rejected their submission, likely due to insufficient data support.
Science, a comprehensive scientific journal and leading publication in the US mainly focused on publishing important original scientific research and reviews, is one of the most authoritative academic journals worldwide.
Sun said that in multinational studies, especially those involving behavior, the research process needs to be simple, transparent and objective. This means the experimental procedures should be the same in different cities.
Even with significant financial support, it is difficult for a research team to avoid flaws when replicating the same experiment in a number of countries.
“As scholars, when we see research results that are unfavorable to China, we should first remain calm. Emotional outbursts do not have a positive impact,” Sun said.
Compared with STEM (Science, Technology, Engineering and Mathematics) subjects, social sciences in China started later and still have room for improvement in terms of development.
Visibility in the international academic community is insufficient, and scholars hope to gradually improve this situation through the efforts of the academic community, Sun said.
He added: “We initially contacted scientific journals after sensing that Chinese people had been offended. As the experiments progressed, we had more and more evidence to support our views, and our statements became more objective. Dialogues in academic journals are based on evidence and logic.”
In 2019, in a WeChat group called Behavioral Economics that comprised more than 200 scholars, including Zhang Qi, a professor at Old Dominion University, issued a call for group replication studies.
The call drew a quick response from over 10 other university professors, including Yang and Sun, who formed a WeChat group called Captain China.
Yang said: “To restore truth and correct misunderstandings about China’s integrity in the outside world, it is necessary to present complete and rigorous research. For scholars, this is a duty that cannot be shirked.”
The research team conducted a “lost wallet” experiment at 500 locations in 10 cities, including Beijing, and cities in Zhejiang, Guangdong, Shaanxi and Heilongjiang provinces.
In addition to assessing whether staff members who found the wallets would contact the owner via email, an important criterion was added — whether the wallet would be retrieved in other ways.
The experiment found that while only 27.4 percent of wallet owners were contacted by email, 77.8 percent of the lost wallets were successfully returned. The important indicator of “wallet recovery rate” proved that Chinese citizens have a high level of integrity.
The team set up various sampling points in each city included in the study, with the type of wallet, business cards, owner’s name, and where the wallet was lost all consistent with Cohn’s experiment.
In addition to the eight cities chosen by Cohn, including Beijing and Shanghai, the team added Harbin, Heilongjiang province, and Nanjing, Jiangsu province, to enlarge the samples.
In Cohn’s experiment, the foreign research assistants handed a “found” wallet to staff members at the location and left immediately after giving a brief explanation.
If no email was received, it was not known how the wallet would subsequently be dealt with.
Kong Shaonan, a doctoral student involved in conducting the Chinese experiment, said: “The Chinese subjects chosen by Cohn, such as employees at scenic spots, hotel workers and delivery personnel, may be more inclined to use other methods of communication such as WeChat or phoning. They might also choose to hand in a wallet at a police station, or keep it for the owner to collect. Obviously, the authors did not consider this cultural difference.”
The Chinese research team decided to divide its study into two groups. The first group of observers recorded on-site information and then left, while the second group recorded the entire delivery process. Each group was equipped with hidden cameras to ensure data accuracy.
They also returned to the locations of the experiments to retrieve the wallets, which was not the case in Cohn’s research.
Yang said, “These study designs were the result of intense discussions with experts from fields such as public health, psychology and economics. The procedures for the experiments were very rigorous.”
The experiments found a significant negative correlation between the “email response rate” and the “national collectivism index”.
In other words, assessing integrity requires different cultural influences to be considered. In collectivist cultures, people tend to keep a wallet for the owner, or hand it to relevant authorities, while in individualistic cultures, people tend to actively contact the owner.
Yang said there has been bias against China from some scholars in the international academic community, which can also be seen in study design, methodology and analysis in other research.
For example, to assess honesty, the 2016 Study of Honesty Levels in 15 Countries used experiments involving tossing a coin and answering questions.
Participants were asked to toss a coin, report whether it landed heads or tails, and then answer three complex questions. Those who reported the coin landed heads more than 50 percent of the time and answered more than one question correctly were considered “dishonest”.
Yang said this study had obvious flaws in its methodology, as it did not differentiate between attitudes toward tossing a coin in Eastern and Western cultures.
In the rankings, East Asian countries scored lower than Western nations, with China ranked the “least honest”.
“This research was widely publicized by the media in various countries, significantly affecting East Asian culture and China’s international image. However, due to a lack of timely, scientific large-scale replication experiments, and the absence of new theoretical constructions, the opportunity to set the record straight for East Asia, especially China, was lost,” Yang added.
After the Chinese group’s paper was published in Proceedings of the National Academy of Sciences, Cohn and his research team sent a letter to PNAS, and the two teams had further talks via the journal.
The new round of discussion was a process of mutual encouragement and inspiration between scholars.
The two rounds of debate not only restored the credibility of Chinese citizens, but also objectively promoted research on integrity in the international academic community, deepening the understanding of integrity norms and behavior across different cultural backgrounds.
Sun said: “It is not necessary to blindly believe in the research conclusions in scientific journals. At the same time, we believe that dialogue is better than confrontation. Scientific and rigorous data and logic are more effective in clarifying facts than emotional venting on message boards.”
He added that as China becomes increasingly powerful, the academic community is now more aware of the importance of developing social sciences, and that there is increasingly good policy support for scholars to conduct better research.
For example, Sun said completion of the research on civic honesty was also due to support from the National Natural Science Foundation of China for his and several other participants’ funding projects.
Since 2019, the foundation has carried out a pilot reform for the Outstanding Young Talents Program.
“These projects centered on talent encourage social science researchers to explore freely, and give researchers a great deal of autonomy and enthusiasm,” Sun said.