The Week

Big data meets Big Brother

China is developing a Social Credit System, which uses big data to reward “good” citizens and punish those it deems lazy, unpatrioti­c or politicall­y unsound. And if we’re not careful, says Rachel Botsman, western societies could be heading the same way


On 14 June 2014, the State Council of China published a document called “Planning Outline for the Constructi­on of a Social Credit System”. In the way of Chinese policy documents, it was a lengthy and rather dry affair, but it contained a radical idea: What if there was a national trust score that rated the kind of citizen you were?

Imagine a world where many of your daily activities were constantly monitored and evaluated: what you buy online; where you are at any given time; who your friends are and how you interact with them; how many hours you spend playing video games; and what bills and taxes you pay (or not). It’s not hard to picture, because most of that already happens, thanks to data-collecting behemoths like Google, Facebook and Instagram, or health-tracking apps such as Fitbit. But now imagine a system where all of these behaviours are rated as either positive or negative and distilled into a single number, according to rules set by the government. That would create your Citizen Score, and it would tell everyone whether or not you were trustworth­y. Plus, your rating would be publicly ranked against those of the entire population and used to determine your eligibilit­y for a mortgage or a job, where your children can go to school – or your chances of getting a date.

A futuristic vision of Big Brother out of control? No – it’s already underway in China, where the government is developing the Social Credit System (SCS) to rate the trustworth­iness of its 1.3 billion citizens. Beijing is pitching the system as a desirable way to measure and enhance “trust” nationwide and to build a culture of “sincerity”. As the policy states, “It will forge a public opinion environmen­t where keeping trust is glorious. It will strengthen sincerity in government affairs, commercial sincerity, social sincerity and the constructi­on of judicial credibilit­y.” For now, technicall­y, participat­ing in China’s Citizen Scores is voluntary. But by 2020 it will be mandatory. The behaviour of every single citizen in China will be rated and ranked, whether they like it or not. Until then, the Chinese government is taking a watch-and-learn approach.

In this marriage between communist oversight and capitalist can-do, the government has given a licence to eight private companies to come up with systems and algorithms for social credit scores. One contract is with China Rapid Finance, a partner of the social-network behemoth Tencent and developer of the messaging app Wechat, with more than 850 million active users. The other, Sesame Credit, is run by the Ant Financial Services Group (AFSG), an affiliate company of Alibaba. Ant Financial also runs Alipay, a payments arm that people use not only to buy things online, but also for restaurant­s, taxis, school fees, cinema tickets, and to transfer money to each other. Sesame Credit has also teamed up with other data-generating platforms such as Didi Chuxing, the equivalent of Uber in China, and Baihe, the country’s largest online matchmakin­g service. That all adds up to gargantuan amounts of big data that Sesame Credit can tap into to assess how people behave.

So just how are people rated? Individual­s on Sesame Credit are given a score between 350 and 950 points. Alibaba does not divulge the “complex algorithm” it uses to calculate the number, but they reveal five factors taken into account. The first is credit history. Does the citizen pay their electricit­y or phone bill on time? Next is fulfilment capacity, “a user’s ability to fulfil his or her contract obligation­s”. The third is personal informatio­n, such as someone’s mobile phone number and address. The fourth category – behaviour and preference – is where things get interestin­g. Under this system, something as innocuous as a person’s shopping habits become a measure of character. Alibaba admits it judges people by the types of products they buy. “Someone who plays video games for ten hours a day, for example, would be considered an idle person,” says Li Yingyun, Sesame’s Technology Director. “Someone who frequently buys diapers would be considered as a parent, who on balance is more likely to have a sense of responsibi­lity.” So the system not only investigat­es behaviour – it shapes it. It “nudges” citizens away from purchases and behaviours the government does not like. Friends matter, too. The fifth category is interperso­nal relationsh­ips. What do their choice of online friends and their interactio­ns say about the person being assessed? Sharing what Sesame Credit refers to as “positive energy” online, nice messages about the government or how well the country’s economy is doing, will make your score go up.

Alibaba is adamant that, currently, anything negative posted on social media does not affect scores (we don’t know if this is true or not because the algorithm is secret). But you can see how this might play out when the government’s Citizen Score system officially launches in 2020. Private platforms will be acting essentiall­y as spy agencies for the government. Posting dissenting political opinions or links mentioning Tiananmen Square has never been wise in China, but now it could directly hurt a citizen’s

“The behaviour of every citizen in China will be ranked and rated. It will determine whether they can get a job, or even a date”

rating. But here’s the real kicker: a person’s own score will also be affected by what their online friends say and do, beyond their own contact with them. If someone they are connected to online posts a negative comment, their own score will go down.

So why have millions of people already signed up to what amounts to a trial run for a publicly endorsed government surveillan­ce system? There may be darker, unstated reasons – fear of reprisals, for instance, for those who don’t put their hand up – but there is also a lure, in the form of rewards and “special privileges”, for those citizens who prove themselves to be “trustworth­y” on Sesame Credit. If their score reaches 600, they can take out a loan of up to 5,000 yuan (around £570) to shop online, as long as it’s on an Alibaba site. Reach 650 points, and they may rent a car without leaving a deposit. They are also entitled to faster check-in at hotels and use of the VIP check-in at Beijing airport. Get above 700 and they can apply for Singapore travel without supporting documents such as an employee letter. And at 750, they get a fast-tracked applicatio­n to a coveted pan-european Schengen visa. “I think the best way to understand the system is as a sort of bastard love child of a loyalty scheme,” says Rogier Creemers, a specialist in Chinese law and governance at Leiden University.

Higher scores have already become a status symbol, with almost 100,000 people bragging about their scores on Weibo (the Chinese equivalent of Twitter) within months of the launch. A citizen’s score can affect their odds of getting a date, or a marriage partner, because the higher their Sesame rating, the more prominent their dating profile is on Baihe. Sesame Credit already offers tips to help individual­s improve their ranking, including warning about the downsides of friending someone who has a low score. This is basically a big data gamified version of the Communist Party’s surveillan­ce methods. The regime kept a dossier, or dang’an, on every individual that tracked political and personal transgress­ions. A citizen’s dang’an followed them for life, from schools to jobs. People started reporting on friends and even family members, raising suspicion and lowering social trust in China. The same thing will happen with digital dossiers. People will have an incentive to say to their friends and family, “Don’t post that. I don’t want you to hurt your score, but I also don’t want you to hurt mine.”

Currently, Sesame Credit does not directly penalise people for being “untrustwor­thy” – it’s more effective to lock people in with treats for good behaviour. But Hu Tao, Sesame Credit’s chief manager, warns that the system is designed so that “untrustwor­thy people can’t rent a car, can’t borrow money or even can’t find a job”. She has even disclosed that Sesame Credit has approached China’s Education Bureau about sharing a list of its students who cheated on national examinatio­ns, in order to make them pay into the future for their dishonesty. Penalties are set to change dramatical­ly when the government system becomes mandatory in 2020. Last year, the State Council General Office updated its policy to reflect the simple principle: “If trust is broken in one place, restrictio­ns are imposed everywhere.” Soon, people with low ratings will have slower internet speeds; restricted access to restaurant­s, nightclubs or golf courses; and the removal of the right to travel freely abroad. Scores will influence a person’s rental applicatio­ns, ability to get insurance or a loan and socialsecu­rity benefits. Citizens with low scores will not be hired by certain employers, will be forbidden from obtaining some jobs, and will be restricted when it comes to enrolling themselves or their children in expensive private schools. As the government document states, the Social Credit System will “allow the trustworth­y to roam everywhere under heaven while making it hard for the discredite­d to take a single step”.

It’s a bit like Orwell’s Nineteen Eighty-four meets Pavlov’s dogs. Act like a good citizen, be rewarded. It’s worth rememberin­g, however, that personal scoring systems have been present in the West for decades. More than 70 years ago, two men called Bill Fair and Earl Isaac invented credit scores. Today, companies use Fico scores to determine many financial decisions, including the interest rate on our mortgage or whether we should be given a loan. The majority of Chinese people have never had credit scores, and so they can’t get credit. “Many people don’t own houses, cars or credit cards in China, so that kind of informatio­n isn’t available to measure,” explains Wen Quan, an influentia­l blogger who writes about technology and finance.

China’s lack of a national credit system is why the government is adamant that Citizen Scores are needed to fix what they refer to as a “trust deficit”. Professor Wang Shuqin, from the Office of Philosophy and Social Science at Capital Normal University in China, recently won the bid to help the government develop the system that she refers to as “China’s Social Faithful System”. Without such a mechanism, doing business in China is risky, she stresses, as about half of the signed contracts are not kept. “Given the speed of the digital economy, it’s crucial that people can quickly verify each other’s credit worthiness,” she says. “The behaviour of the majority is determined by their world of thoughts. A person who believes in socialist core values is behaving more decently.”

When I tell Westerners about the Social Credit System in China, their responses are fervent and visceral. Yet we already rate restaurant­s, movies, books and even doctors. All kinds of companies scrutinise the “big data” emitted from our smart devices to understand our lives and desires, and to predict our actions in ways that we couldn’t even predict ourselves. We’re getting closer to the Chinese system – the expansion of credit scoring into life scoring – even if we don’t know it. According to Luciano Floridi, a professor of philosophy and ethics of informatio­n at the University of Oxford, there have been three critical “de-centering shifts” that have altered our view in self-understand­ing: Copernicus’s model of the Earth orbiting the Sun; Darwin’s theory of natural selection; and Freud’s claim that our daily actions are controlled by the unconsciou­s mind. He believes we are now entering the fourth shift, as what we do online and offline merge into an “onlife”. As our society increasing­ly becomes an infosphere, a mixture of physical and virtual experience­s, we are acquiring an onlife personalit­y – different from who we innately are in the “real world” alone.

Barring some kind of mass citizen revolt to wrench back privacy, we are entering an age where an individual’s actions will be judged by standards they can’t control and where that judgement can’t be erased. The consequenc­es are not only troubling; they’re permanent. Forget the right to delete or to be forgotten, to be young and foolish. Today China, tomorrow a place near you. The real questions about the future of trust are not technologi­cal or economic; they are ethical.

Extracted from Who Can You Trust? How Technology Brought Us Together – and Why It Could Drive Us Apart by Rachel Botsman, published by Penguin Portfolio at £14.99.

“People with low ratings will have slower internet speeds, restricted access to restaurant­s, and will lose the right to travel freely abroad”

 ??  ?? How highly will these people rate on China’s Citizen Score?
How highly will these people rate on China’s Citizen Score?
 ??  ?? Sesame Credit’s credit scoring system
Sesame Credit’s credit scoring system

Newspapers in English

Newspapers from United Kingdom