EU’s landmark regulation of AI risks compromising public safety

The draft regulations, passed by the European Parliament on Wednesday (June 14), would introduce a full ban on AI for biometric surveillance, emotion recognition and predictive policing.

The unprecedented restrictions on how companies use AI could shape global standards for its adoption.

While the Act will cover EU citizens and companies doing business in the EU, many experts agree they are likely to have far-reaching consequences, as was the case when the EU introduced data regulations in 2018. The proposals are also likely to inform and influence the UK in its own strategic approach to the use of AI.

Indeed, the European Parliament says the AI Act “will set the tone worldwide in the development and governance of AI”.

Tony Porter, who served as the Surveillance Camera Commissioner for England and Wales from 2014 to 2020, said: “Today’s vote in the EU Parliament effectively amounts to a blanket ban on the use of facial recognition technology in public places.

“That puts it at odds with the smarter and more nuanced position of the European Commission, which seeks to regulate the development and use of the technology by putting effective guardrails around it.

“AI and facial recognition technology requires clear legislation and mandatory restrictions that govern its responsible, ethical and proportionate use. Yet to ban it in all circumstances is to protect known criminals and to endanger innocent civilians. There is clear public support for the controlled use of facial recognition technology to tackle serious crime, save lives and make our communities safer.”

Mr Porter, now chief privacy officer at Corsight, added: “As we now enter trilogue discussions between the EU Commission, Parliament and Council, I hope we’ll see a recognition that the way forward is to apply the appropriate safeguards.

“Talk of bans is easy, but regulation is the responsible approach.”

The new rules follow a risk-based approach and establish obligations for providers and those deploying AI systems depending on the level of risk the AI can generate.

AI systems with an unacceptable level of risk to people’s safety would therefore be prohibited, such as those used for social scoring (classifying people based on their social behaviour or personal characteristics).

MEPs have expanded the list to include bans on intrusive and discriminatory uses of AI, such as:

• Real-time’ remote biometric identification systems in publicly accessible spaces;
• Biometric categorisation systems using sensitive characteristics, for example, gender, race, ethnicity, citizenship status, religion or political orientation;
• Predictive policing systems (based on profiling, location or past criminal behaviour);
• Emotion recognition systems in law enforcement, border management, the workplace, and educational institutions; and
• Untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases (violating human rights and right to privacy).

Some exceptions may be allowed. For instance, ‘post’ remote biometric identification systems for law enforcement, where identification occurs after a significant delay, will be allowed to prosecute serious crimes but only after court approval.

“Post” remote biometric identification systems, with the only exception of law enforcement for the prosecution of serious crimes and only after judicial authorization;

MEPs have also ensured the classification of high-risk applications will now include AI systems that pose significant harm to people’s health, safety, fundamental rights or the environment.

AI systems used to influence voters and the outcome of elections and in recommender systems used by social media platforms (with more than 45 million users) were also added to the high-risk list.

Providers of foundation models – a new and fast-evolving development in the field of AI – would now have to assess and mitigate possible risks (to health, safety, fundamental rights, the environment, democracy and rule of law) and register their models in the EU database before their release on the EU market.

Generative AI systems based on such models, such as ChatGPT, would have to comply with transparency requirements (disclosing that the content was AI-generated, also helping distinguish so-called deep-fake images from real ones) and ensure safeguards against generating illegal content. Detailed summaries of the copyrighted data used for their training would also have to be made publicly available.

To boost AI innovation and support SMEs, MEPs added exemptions for research activities and AI components provided under open-source licences. The new law promotes so-called ‘regulatory sandboxes’, or real-life environments, established by public authorities to test AI before it is deployed.

Finally, MEPs want to boost citizens’ right to file complaints about AI systems and receive explanations of decisions based on high-risk AI systems that significantly impact their fundamental rights.

MEPs also reformed the role of the EU AI Office, which would be tasked with monitoring how the AI rulebook is implemented.

After the vote, co-rapporteur Brando Benifei (S&D, Italy) said: “All eyes are on us today. While big tech companies are sounding the alarm over their own creations, Europe has gone ahead and proposed a concrete response to the risks AI is starting to pose.

“We want AI’s positive potential for creativity and productivity to be harnessed but we will also fight to protect our position and counter dangers to our democracies and freedoms during the negotiations with Council.”

Co-rapporteur Dragos Tudorache (Renew, Romania) said: “The AI Act will set the tone worldwide in the development and governance of artificial intelligence, ensuring that this technology, set to radically transform our societies through the massive benefits it can offer, evolves and is used in accordance with the European values of democracy, fundamental rights, and the rule of law.”

Talks will now begin with EU countries in the Council on the final form of the law.

The aim is to reach an agreement by the end of this year.