EU: Artificial Intelligence Regulation Should Protect People’s Rights

(Brussels, July 12, 2023) – On July 12, 2023, Human Rights Watch joined 149 other civil society organizations to urge the EU to maintain and enhance human rights protections in the upcoming negotiations on the regulation of artificial intelligence. The statement urges the EU to prohibit harmful and discriminatory surveillance and other abusive applications of AI, mandate fundamental rights impact assessments during the development and before the deployment of AI systems, and provide effective remedies for people harmed by AI, among other safeguards. The statement is reproduced below and can be accessed along with a list of signatories here.

EU Trilogues: The AI Act must protect people’s rights

A civil society statement on fundamental rights in the EU Artificial Intelligence Act

As European Union institutions[1] begin trilogue negotiations, civil society calls on EU institutions to ensure the Regulation puts people and fundamental rights first in the Artificial Intelligence Act (AI Act).

In Europe and around the world, AI systems are used to monitor and control us in public spaces, predict our likelihood of future criminality, facilitate violations of the right to claim asylum, predict our emotions and categorise us, and to make crucial decisions that determine our access to public services, welfare, education and employment.

Without strong regulation, companies and governments will continue to use AI systems that exacerbate mass surveillance, structural discrimination, centralised power of large technology companies, unaccountable public decision-making and environmental damage.

We call on EU institutions to ensure that AI development and use is accountable, publicly transparent, and that people are empowered to challenge harms:

1.Empower affected people with a framework of accountability, transparency, accessibility and redress

It is crucial that the EU AI Act empowers people and public interest actors to understand, identify, challenge and seek redress when the use of AI systems exacerbate harms and violates fundamental rights. To do this, it is crucial that the AI Act develops a framework of accountability, transparency, accessibility and redress. This must include:

• An obligation on all public and private ‘users’ (deployers) to conduct and publish a fundamental rights impact assessment before each deployment of a high-risk AI system and meaningfully engage civil society and affected people in this process;
• Require all users of high-risk AI systems, and users of all systems in the public sphere, to register their use in the publicly viewable EU database before deployment;
• Ensure that EU-based AI providers whose systems impact people outside of the EU are subject to the same requirements as those inside the EU.
• Ensure horizontal and mainstreamed accessibility requirements for all AI systems;
• Ensure people affected by AI systems are notified and have the right to seek information when affected by AI-assisted decisions and outcomes;
• Include a right for people affected to lodge a complaint with a national authority if their rights have been violated by the use of an AI system;
• Include a right to representation of natural persons and the right for public interest organisations to lodge standalone complaints with a national supervisory authority;
• Include a right to effective remedies for the infringement of rights.

2. Draw limits on harmful and discriminatory surveillance by national security, law enforcement and migration authorities

Increasingly AI systems are developed and deployed for harmful and discriminatory forms of state surveillance. Such systems disproportionately target already marginalised communities, undermine legal and procedural rights, as well as contributing to  mass surveillance. When AI systems are deployed in the context of law enforcement, security and migration control, there is an even greater risk of harm, and  violations of fundamental rights and the rule of law. To maintain public oversight and prevent harm, the EU AI Act must include:

• A full ban on real-time and post remote biometric identification in publicly accessible spaces, by all actors, without exception;
• A prohibition of all forms of predictive and profiling systems in law enforcement and criminal justice (including systems which focus on and target individuals, groups and locations or areas);
• Prohibitions on AI in migration contexts to make individual risk assessments and profiles based on personal and sensitive data, and predictive analytic systems when used to interdict, curtail and prevent migration;
• A prohibition on biometric categorisation systems that categorise natural persons according to sensitive or protected attributes as well as the use of any biometric categorisation and automated behavioural detection systems in publicly accessible spaces;
• A ban on the use of emotion recognition systems to infer people’s emotions and mental states;
• Reject the Council’s addition of a blanket exemption from the AI Act of AI systems developed or used for national security purposes;
• Remove exceptions and loopholes for law enforcement and migration control introduced by the Council;
• Ensuring public transparency as to what, when and how public actors deploy high-risk AI in areas of law enforcement and migration control, avoiding any exemption to the obligation to register high-risk uses into the EU AI database.

3. Push back on Big Tech lobbying: remove loopholes that undermine the regulation

The EU AI Act must set clear and legally-certain standards of application if the legislation is to be effectively enforced. The legislation must uphold an objective process to determine which systems are high-risk, and remove any ‘additional layer’ added to the high-risk classification process. Such a layer would allow AI developers, without accountability or oversight, to decide whether or not their systems pose a ‘significant’ enough risk to warrant legal scrutiny under the Regulation. A discretionary risk classification process risks undermining the entire AI Act, shifting to self-regulation, posing insurmountable challenges for enforcement and harmonisation, and incentivising larger companies to under-classify their own AI systems.

Negotiators of the AI Act must not give in to lobbying efforts of large tech companies seeking to circumvent regulation for financial interest. The EU AI Act must:

• Remove the additional layer added to the risk classification process in Article 6 restore the clear, objective risk-classification process outlined in the original position of the European Commission;
• Ensure that providers of general purpose AI systems are subject to a clear set of obligations under the AI Act, avoiding that smaller providers and users bear the brunt of obligations better suited to original developers.