The EU is funding dystopian Artificial Intelligence projects
Discussions on the negative impact of Artificial Intelligence in society include horror stories plucked from either China’s high-tech surveillance state and its use of the controversial social credit system, or from the US and its use of recidivism algorithms and predictive policing.
Typically, Europe is excluded from these stories, due to the perception that EU citizens are protected from such AI-fueled nightmares through the legal protection offered by the GDPR, or because there is simply no horrorinducing AI deployed across the continent.
In contrast to this perception, journalists and NGOs have shown that imperfect and ethically questionable AI systems such as facial recognition, fraud detection and smart (a.k.a surveillance) cities, are also in use across Europe. For example, the UK police are implementing facial recognition to monitor protests and soccer matches; the Dutch government is being sued for SyRI, a risk-scoring algorithm that is targeting the poor; and the Polish Ministry of Labour and Social Policy introduced a controversial system that profiles unemployed people to determine the type of assistance that a person can obtain from local labour offices.
Meanwhile, AI systems like these are on track to proliferate this decade: one of the three ‘pillars’ of the European Commission’s plan on AI is to boost “AI uptake across the economy, both by the private and public sector.”
To that end, the Commission is investing in the development of AI systems through funding programs such as Horizon 2020, which will have invested nearly €80 billion of funding over 7 years (2014 to 2020), with a significant portion of that going to so-called ‘artificial intelligence’ projects.
According to last week’s leaked white paper on AI regulation, there are plans to increase this funding and further invest in “targeted cloud-based artificial intelligence services,” to “offer world-leading master programs in artificial intelligence,” and to ensure “access to finance for artificial intelligence innovators.”
Amid this proliferation, some EU residents might be comforted by the Commission’s stated commitment to ‘Trustworthy AI,’ most notably through its Ethics Guidelines for Trustworthy AI and the potential influence they might have on the fabled ‘AI Regulation’ promised to come in the first 100 days of the new Commission mandate.
Indeed, it seems clear that public procurement in general, and these EU funding mechanisms in particular, have huge potential to promote the development of ‘trustworthy’ AI systems: i.e. that respect human rights, adhere to ethical guidelines and promote human agency.
Despite the EU’s commitment to ‘trustworthy’ AI sounding noble, the history of technological investments made under Horizon 2020 casts doubt on these intentions.
Take for example iBorderCtrl, a Horizon 2020-funded project that aims to create an automated border security system to detect deception based on facial recognition technology and the measurement of micro-expressions. In short, the EU spent €4.5 million on a project that ‘detects’ whether a visitor to the continent is lying or not by asking them 13 questions in front of a webcam.
The historical practice of lie detection is lacking in substantial scientific evidence and the AI technologies being used here to analyse micro expressions are just as questionable.
To make matters worse, the Commission is ignoring the transparency criteria outlined in the Ethics Guidelines by refusing to publish certain documents, including an ethics assessment, “on the grounds that the ethics report and PR strategy are “commercial information” of the companies involved and of “commercial value”.”
Another example of untrustworthy AI funded by Horizon 2020 is the SEWA project. This project received €3.6 million to develop technology that can read the depths of human sentiment and emotions ‘in the wild’ — for the ultimate purpose of more effectively marketing products to consumers using an ad recommendation engine.
Indeed, the SEWA project was singled out by Shoshana Zuboff in her book, The Age of Surveillance Capitalism, as a pioneer in techniques harvesting personal data to be processed for behavioural prediction/manipulation.
Not only is there doubt among the scientific community that human emotions can be reliably inferred from biometric analysis, but one can also question how ‘trustworthy’ is it to develop technology that exploits people’s emotional states for commercial gain.
In the leaked White Paper, the Commission is caught in a dilemma about whether or not to introduce a 3-5 year ban on facial recognition: on the one hand, such a ban would allow time to safeguard against any abuse, but on the other, they worry that a ban “might hamper the development and uptake of this technology.”
Just recently, the US’s Chief Technology Officer Michael Kratsios urged the EU to “avoid heavy-handed innovation-killing models” in its AI regulation, but we must ask seriously whether certain forms of innovation – in the domain of mass surveillance and pseudoscientific lie detection – may not require a heavy hand.
Some uses of AI could be by definition untrustworthy, and we need to see a commitment from the EU not to blindly promote ‘AI uptake’ without consideration of its impact.
Projects such as iBorderCtrl and SEWA lead to larger questions: if the criteria to fund AI is not based on a technology’s trustworthiness, ethics, or innovative capabilities, then which drivers inform these decisions?