Big Brother comes to high school
HANGZHOU, CHINA— At first, it just seemed cool.
When facial recognition cameras were installed at a century-old high school here in eastern China, students got in and out of campus, picked up lunch, borrowed books and even bought drinks from a vending machine just by peering into the cameras.
No more worrying about forgetting to carry your ID card.
But last March, the cameras appeared in some classrooms — and they did a lot more than just identify students and take attendance.
Using the latest artificial intelligence software, the devices tracked students’ behaviour and read their facial expressions, grouping each face into one of seven emotions: anger, fear, disgust, surprise, happiness, sadness and what was labelled as neutral.
The cameras took the data on individual facial expressions and used that information to create a running “score” on each student and class. If a score reached a predetermined point, the system triggered an alert. Teachers were expected to take action: to talk to a student perceived to be disengaged, for example, or overly moody.
School administrators reckoned the data could provide feedback for teachers as well, about their lectures and classroom management, though they spoke of no immediate plans to use those details as part of their evaluations.
Most students came to hate the constant monitoring — and the consequences that followed when the machines reported scores suggesting individuals or entire classes weren’t paying attention. Some students went so far as to figure out how to game the system by feigning what the cameras’ designers wanted to see.
“If you feel angry, you need to control yourself,” said Zhu Juntao, 17, using his two forefingers to press up the ends of his mouth, as if smiling.
Parents had mixed reactions, but enough of them complained about what they saw as an intrusion on privacy that school administrators have hit the pause button on the cameras.
Not that those officials have given up on the system. It just needs further studying and some tweaking, says Zhang Guanchao, the school’s deputy principal, who believes it’s a useful tool for teachers.
Facial identifying technology has been developing rapidly and is being deployed in more places around the world.
But no country has been employing facial recognition as aggressively as China. That reflects the central government’s intense focus on public security and monitoring of residents, particularly in China’s far west Xinjiang region, where Beijing is using highly sophisticated facial recognition, iris scanners and other artificial intelligence software to keep tabs on — and watch for any separatist activities from — its Muslim Uighur population.
At the same time, Beijing is making a big push in artificial intelligence. China has set a goal of being the world’s AI leader by 2030 and is investing heavily to support startups, research and more use of smart surveillance technologies. State media said recently that Beijing’s subway system plans to install facial recognition cameras along with palm scanners this year.
In Beijing and throughout China, closed-circuit cameras and other surveillance devices are so ubiquitous that they’ve become part of the landscape. If facial recognition helps with public safety, some say, that’s a good thing.
“Perhaps people would behave themselves more,” said Xia Chuzi, a 19-yearold student interviewed in Beijing.
Chen Hong, another Beijing resident, said his main worry is whether AI technology will work properly in identifying faces correctly. “I’m not concerned about privacy,” said the 24-year-old, who installs high-speed internet equipment for a living.
Hangzhou, a top tourist destination about 100 miles southwest of Shanghai, is now one of the country’s leading tech hubs, thanks in part to e-commerce giant Alibaba. Also based in the city is Hikvision, the world’s largest maker of video surveillance products.
Hikvision supplied the face-recognition devices to Hangzhou No. 11 High School. The partially state-owned company did not respond to requests for an interview.
Experts say technologies recognizing or verifying faces is one thing, but monitoring emotions with AI devices takes it to a whole other level. They include not just cameras but hats and caps with sensors to monitor brain waves that detect shifts in a person’s mood.
Human rights and privacy advocates see such emotional surveillance as part of China’s widening security control regime, an increasingly Orwellian world in which people can’t escape the eye of government.
“It’s an incredibly dangerous precedent to affix somebody’s behaviour or certain actions based on emotions or characteristics presented in their face,” said Clare Garvie, an associate at the Center on Privacy and Technology at Georgetown University Law Center.
Educators in China have been sharply critical of the Hangzhou school, not only for invading students’ privacy — neither they nor their parents were asked to give consent — but for charging ahead with a unproven system that purports to improve student performance.
Even assuming the machines can accurately read facial emotions, it’s far from clear how outward expressions are related to learning, says He Shanyun, an associate professor of education at Zhejiang University in Hangzhou. She thinks facial recognition is flawed in another way: It doesn’t account for different personalities and a Chinese culture that may be predisposed to a stoic face.
Zheng Suning, a 10-grader at Hangzhou No. 11, speaks proudly of her school. It was founded in1904, but is now one of the most high-tech in the country. “We have visitors regularly,” she said. School administrators, however, declined a request for a tour.
Zheng recalls the trouble when she had misplaced her school ID. “You show your face to the machine, and they bring out your lunch tray,” she said.
Still, the 16-year-old dreads a return of the AI cameras in the classroom. Like most other high school students in China, Zheng and her classmates are in school from early morning to late at night. On top of that, Zheng takes private tutorial lessons twice a week. She says maybe the cameras will help her be a better student, but she worries they will add more stress. She doesn’t know how she could avoid not looking sleepy. “I’m already so tired,” Zheng said. Facial recognition cameras haven’t been installed in every classroom at the school yet. And they monitored only 10th-graders — and only for about two months before their use was suspended. Educators worry the emotion-monitoring will encourage excessive attention on outward behaviour or become an active means of social control. That’s partly why Xiong Bingqi, an education professor at Shanghai Jiaotong University, calls it “black technology.”
“The cameras have a very bad influence on students’ development,” he said.