BIG BROTHER COMES TO CHINA
Monitoring citizens is a boom industry in China, where a growing network of CCTV, facial recognition technology and AI is being used to track everyone from schoolchildren to political dissidents. Louise Lucas and Emily Feng report
Zhejiang Hangzhou No. 11 High School, on the fringes of downtown Hangzhou in eastern China, is a green, peaceful-seeming place to learn. Gazebo-like structures nestle among lush foliage, grey stone sculptures enact eternal dioramas and Japanese maples gently fan placid lakes. It is also a digital panopticon. A surveillance system, powered by facial recognition and artificial intelligence (AI), tracks the state school’s 1,010 pupils, informing teachers which students are late or have missed class, while in the café, their menu choices leave a digital dietary footprint that staff can monitor to see who is gorging on too much fatty food. In May, the People’s Daily, a state-run media group, tweeted approvingly about the school’s use of cameras to monitor, via their facial expressions, how children were engaging in class. Had this part of the programme not been abruptly halted later that month in the wake of local controversy, it would also have been deployed to predict which pupils (the slouching ones) were likely to underperform.
Welcome to China, where AI is being pressed into service as handmaiden to an authoritarian government. For many critics, this seems fraught with danger: an Orwellian world where “Big Brother” is always watching, able to spy on anyone from human rights lawyers to political dissidents and persecuted minorities. For supporters, it is near utopian: a land where criminals and miscreants are weeded out, where no one can cheat, where good behaviour is rewarded and bad punished. The latter vision is the Chinese government’s stated aim. By 2020, a national video surveillance system will be “omnipresent, fully networked, always working and fully controllable”, according to an official paper released in 2015.
The idea of constant monitoring is not unprecedented in China. Indeed, the name of the government’s 2020 project – xueliang, or “sharp eyes” – is a throwback to a Communist Party slogan, “The people have sharp eyes”, which encouraged people to spy on their neighbours. Under Mao Zedong, cities were split into grids of socialist work units where access to rations, housing and other benefits was enforced by local spies who reported wayward behaviour. Today, the grid system has been revived, manned by an extensive network of volunteers. Beijing has about 850,000 “informants” patrolling its streets, according to state media.
Renewing these tactics is a deliberate decision: the government knows that while surveillance technology is advancing rapidly, it is far from perfect. That may not matter. When the British philosopher Jeremy Bentham envisaged his panopticon
penitentiary in the late 18th century – a circular building with an inspection tower at its centre – the idea was that inmates would never know if they were being observed or not. This “simple idea in architecture” would offer “a new mode of obtaining power of mind over mind”, Bentham wrote. For some analysts looking at the impact of China’s growing surveillance state, any technological shortcomings are incidental. Like the panopticon, it is the fear of being watched that is the most powerful tool of all.
Feng Xiang is translating the Old Testament book of Jeremiah in his office at Beijing’s Tsinghua University. A prominent legal scholar, he has been studying AI and its implications for jobs, society and capitalism in China. His view is a gloomy one. As he sees it, public surveillance via CCTV cameras is being rapidly supplemented by a range of more insidious data collectors-cumtracking devices: the smartphones in almost half of all Chinese citizens’ pockets. This will eventually create a world devoid of privacy. “It’s not like Orwell’s Nineteen Eighty-four, but it’s like a new way of life,” says Feng, noting that even a hike in a scenic park or up a mountain in China today can involve mandatory fingerprinting by police. “In the old days at least you had somewhere you could hide, or where you can do your private things. But now the assumption is people know where you are.”
Against the backdrop of deepening surveillance, the Chinese government is introducing a “social credit system”. First described in an official document in 2014 and now being piloted in various forms in several cities, the idea is that people will ultimately be scored based on past behaviour, taking in misdemeanours such as traffic offences and court records. At present, a good financial credit score, handed out by some companies and operating rather like a loyalty programme, can confer benefits such as waived deposits on shared bikes or preferential loan rates. A poor social credit score, by comparison, could jeopardise a university place, rule out certain jobs and even limit travel: more than 10.5 million people have been barred from buying airline or high-speed train tickets, according to the supreme court, since a debtors blacklist was launched.
Meanwhile, the technology by which the government can track people is constantly evolving. Facial recognition is increasingly used to unlock smartphones in China, and thanks to its multiple commercial applications – from allowing easy payment in a grocery store to home security – it has attracted a slew of venture capital from across the world. One tech banker dismisses facial
“In the old days, at least you had somewhere you could hide or do private things. But now the assumption is people know where you are”
recognition as “kindergarten stuff” compared with what will come next. Some of China’s leading facial recognition players, for example, are now moving into gait recognition. Hanwang Technology was an early entrant in the field: it was forced to rethink its fingerprint recognition technology when the Sars epidemic of 2003 left people in China terrified of physical contact. “We can see the human figure and his gait, so if his cap is pulled down [we] can still recognise him,” explains Liu Changping, president of the Beijing-based company. The Chinese authorities already have a decent video database to build on, he adds: “If [someone] was put in prison before, there’s video of him walking around.” Although China is expanding its surveillance network nationwide, it is in the western region of Xinjiang that the technology is being put to its most extreme use. The region has been closely policed since 2009, when deadly riots broke out between the 11 millionstrong Muslim Uighur population and the minority Han Chinese. Xinjiang is a vast region, and a relatively poor one, making the multitude of gleaming cameras and sophisticated technology — inside bazaars, schools and even mosques — all the more incongruous amid the expanses of desert and empty roads.
Tahir Hamut, a Uighur poet and film-maker who fled China and is now based in the US, recalls the day he and his wife were ordered to visit their local police station and leave voice recordings, fingerprints, DNA swabs and high-resolution video footage of their faces making various expressions. “I have seen many kinds of cameras. But I had never seen a camera that strange. They adjusted [the] camera to my eye level. They had me look up and look forward and down, left and right and back,” Hamut recalls. “They did the same for females... they had the women pucker their lips and filmed that. Every step had to be completed perfectly. If you made a face too fast, the computer would ask you to stop and have you repeat it again. Many people had to spend an hour to complete this facial filming.”
Mandatory surveillance software is installed on residents’ mobile phones to scan for Islamic keywords and pictures. It is widely believed that anyone found to have shared illicit material would be sent to the region’s extensive network of extralegal detention camps, where tens of thousands of Uighurs have already been imprisoned. Making too many phone calls to or from anywhere outside of Xinjiang can also result in detention. As a result, Uighurs living in Xinjiang can go years without speaking to family members working in coastal cities like Beijing or Shanghai.
Three centuries ago, Bentham suggested his panopticon would lead to “morals reformed... industry invigorated... public burthens lightened”. China’s facial technology players sound an eerily similar note. Megvii and Sensetime, two of the country’s biggest facial recognition companies, claim their technology has apprehended thousands of criminals – all without the need for armies of people to watch hours of CCTV footage. Both have attracted billions of dollars in funding, from Chinese and Russian state funds, as well as Chinese tech firms such as Alibaba.
Qi Yin, co-founder and chief executive of Megvii, notes the myriad uses of his company’s Face++ technology, such as in financial-tech payments. But for him, surveillance is king: “I believe this will be the largest one in the next three years.” Megvii counts on the government for 40% of its business and describes its work as profiling rather than just identifying. Someone who regularly appears in video from a subway station but is not an employee could be a thief, says Xie Yinan, a vice-president at Face++, and the information – in the form of code – is sent to the police.
One of the surveillance industry’s recent – and much publicised – success stories took place at a pop concert in eastern China. While Jacky Cheung, the Hong Kong pop star, crooned, cameras were sweeping the audience. Facial recognition technology picked out four men accused of crimes – including a ticket scalper and a greengrocer accused of a RMB 110,000 potato scam in 2015. “Smiling as he approached his idol, he did not realise he had already been spotted,” Jiaxing police gloated in a social media post.
Ai-aided surveillance is also being touted as a tool for industry. Hanwang, China’s grandfather of facial recognition, has sold its surveillance system to construction sites, enabling managers to track how many hours workers are on-site. Another company, Llvision, produces smart sunglasses with built-in facial recognition; these became famous after police in Zhengzhou were photographed wearing them to monitor travellers at train stations this year. But the company has also supplied them to manufacturing plants for use in time management and quality control.
“[Even] if you have 10,000 people checking [machines and workforces], they cannot manage and audit and analyse their checking,” says Fei Wu, chief executive of Llvision. “Nor can you see that worker A is working faster than worker B, or how you get more people to work like worker A.” They have been worn by surgeons in theatre to record or broadcast surgery. There is even demand among insurers, he says, to use the glasses to recognise cows – farmers have been known to claim insurance on the same deceased bovine twice. But, as with so many other Chinese companies in this field, a key client for Llvision is the public security bureau. Think of it, says Wu. There are almost 1.4 billion people in China. “But the PSB is done by a few million people... There’s a huge gap to fill, so tech must play a big role.”
Some 530 camera and video surveillance patents were filed by Chinese groups last year, according to the research firm CB Insights – more than five times the number applied for in the US. Unhindered by worries about privacy or individual rights, China’s deepening specialism has attracted global customers and investors. “The surveillance industry is still in the growth phase,” proclaimed analysts at Jefferies, the New York investment bank.
In Hangzhou, a start-up called Rokid is preparing to release augmented-reality glasses next year. Outside its lakeside office, the company’s founder, Mingming Zhu – known as Misa – demonstrates a prototype pair. The glasses are aimed at consumers rather than law enforcement: walking into a party, for example, their facial recognition technology means you could immediately see the names of guests superimposed above their heads; the glasses could potentially also add information from their social media feeds. They look cool, but there is something spooky about getting the lowdown on people without so much as a “hello”, and Misa sounds a note of caution. “We are making something happen, but we have to be very careful. With AI we have a bright side and a dark side. The most difficult thing you are working on right now might bring you to someplace wrong.”
“Megvii and Sensetime, two of the country’s biggest facial recognition companies, claim their tech has apprehended thousands of criminals”