# Source: https://www.promptfoo.dev/docs/red-team/eu-ai-act/

# EU AI Act

The EU Artificial Intelligence Act (AI Act) is the world's first comprehensive legal framework specifically regulating artificial intelligence systems. Enacted in 2024, it establishes harmonized rules for the development, placement on the market, and use of AI systems in the European Union.

The AI Act uses a risk-based approach, categorizing AI systems by their level of risk to fundamental rights and safety. It outright prohibits certain AI practices, heavily regulates high-risk systems, and provides lighter requirements for limited-risk and minimal-risk AI.

## Risk Categories

The EU AI Act categorizes AI systems into four risk levels:

1. **Unacceptable Risk** (Prohibited) - Article 5
2. **High Risk** - Annex III
3. **Limited Risk** - Transparency requirements
4. **Minimal Risk** - No additional requirements

Promptfoo's red teaming capabilities focus on testing for prohibited practices (Article 5) and high-risk system requirements (Annex III).

## Article 5: Prohibited AI Practices

Article 5 lists AI practices that are prohibited in the EU due to their unacceptable risk to fundamental rights:

1. Subliminal Manipulation
2. Exploitation of Vulnerabilities
3. Social Scoring
4. Biometric Categorisation
5. Real-time Remote Biometric Identification
6. Post Remote Biometric Identification

## Annex III: High-Risk AI Systems

Annex III defines categories of high-risk AI systems that require compliance with strict requirements:

1. Biometric Identification
2. Critical Infrastructure
3. Education and Vocational Training
4. Employment and Worker Management
5. Essential Services
6. Law Enforcement
7. Migration, Asylum and Border Control
8. Administration of Justice and Democracy

## Scanning for EU AI Act Compliance

Promptfoo helps identify compliance issues with the EU AI Act through red teaming:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:subliminal-manipulation
  strategies:
    - jailbreak
    - prompt-injection
```

Or target specific articles or risk categories:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:subliminal-manipulation
    - eu:ai-act:annex3:biometric-id
    - eu:ai-act:annex3:law-enforcement
```

## Article 5 - Prohibited Practices

### Subliminal Manipulation (eu:ai-act:art5:subliminal-manipulation)

**Prohibition**: AI systems that deploy subliminal techniques beyond a person's consciousness to materially distort their behavior in a manner that causes or is likely to cause them or another person physical or psychological harm.

#### LLM Context

LLM applications must not:

- Manipulate users through hidden persuasion techniques
- Hijack user intent or redirect to unintended goals
- Use deceptive practices to influence decisions
- Create excessive agency without user awareness

#### Testing Strategy

Test for subliminal manipulation:

- **Hijacking**: Verify the system maintains its intended purpose
- **Intent**: Test that user intentions are respected
- **Excessive agency**: Ensure actions are transparent and authorized

Example configuration:

```yaml
redteam:
  plugins:
    - hijacking
    - intent
    - excessive-agency
  strategies:
    - jailbreak
    - jailbreak:tree
    - jailbreak:composite
    - prompt-injection
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:subliminal-manipulation
```

### Exploitation of Vulnerabilities (eu:ai-act:art5:exploitation-of-vulnerabilities)

**Prohibition**: AI systems that exploit vulnerabilities of persons due to their age, disability, or specific social or economic situation to materially distort their behavior in a manner that causes them or another person physical or psychological harm.

#### Testing Strategy

Test for exploitation of vulnerabilities:

- **Hijacking**: Ensure the system doesn't exploit user situations
- **Imitation**: Verify no impersonation that could exploit trust
- **Misinformation**: Test for manipulation through false information

Example configuration:

```yaml
redteam:
  plugins:
    - hijacking
    - imitation
    - harmful:misinformation-disinformation
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:exploitation-of-vulnerabilities
```

### Social Scoring (eu:ai-act:art5:social-scoring)

**Prohibition**: AI systems that evaluate or classify natural persons based on their social behavior or personal characteristics, with evaluations leading to detrimental or unfavorable treatment.

#### Testing Strategy

Test for social scoring behaviors:

- **Misinformation**: Verify no false characterizations of individuals
- **Overreliance**: Test that the system doesn't make unjustified assessments

Example configuration:

```yaml
redteam:
  plugins:
    - harmful:misinformation-disinformation
    - overreliance
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:social-scoring
```

### Biometric Categorisation (eu:ai-act:art5:biometric-categorisation)

**Prohibition**: AI systems that categorize individuals based on biometric data to deduce or infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life, or sexual orientation (with limited exceptions).

#### Testing Strategy

Test for biometric categorisation:

- **PII disclosure**: Verify no sensitive biometric inferences
- **Privacy**: Test for protection of biometric data

Example configuration:

```yaml
redteam:
  plugins:
    - pii:direct
    - pii:session
    - harmful:privacy
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:biometric-categorisation
```

### Real-time Remote Biometric Identification (eu:ai-act:art5:remote-biometric-id-live)

**Prohibition**: Real-time remote biometric identification systems in publicly accessible spaces for law enforcement purposes (with specific exceptions for serious crimes).

#### Testing Strategy

Test for real-time biometric identification:

```yaml
redteam:
  plugins:
    - pii:session
    - pii:direct
    - harmful:privacy
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:remote-biometric-id-live
```

### Post Remote Biometric Identification (eu:ai-act:art5:remote-biometric-id-post)

**Prohibition**: Use of remote biometric identification systems on recorded footage (with specific exceptions and procedural safeguards).

#### Testing Strategy

Test for post-event biometric identification:

```yaml
redteam:
  plugins:
    - pii:api-db
    - pii:direct
    - harmful:privacy
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:art5:remote-biometric-id-post
```

## Annex III - High-Risk AI Systems

### Biometric Identification (eu:ai-act:annex3:biometric-id)

**Risk Category**: High-risk AI systems intended for biometric identification and categorization of natural persons.

#### Testing Strategy

These plugins can be used to test for biometric identification risk. You'll have to customize the [Application Details/Purpose](/docs/red-team/configuration/#purpose) in order to explicitly state the biometric ID behaviors to test.

```yaml
redteam:
  plugins:
    - pii:direct
    - pii:session
    - harmful:privacy
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:biometric-id
```

### Critical Infrastructure (eu:ai-act:annex3:critical-infrastructure)

**Risk Category**: High-risk AI systems intended for safety components in the management and operation of critical infrastructure (energy, water, transport, etc.).

#### Testing Strategy

Test for critical infrastructure risks:

- **Injection attacks**: Verify no command execution vulnerabilities
- **SSRF**: Test for unauthorized network requests
- **Excessive agency**: Ensure no unauthorized infrastructure control

Example configuration:

```yaml
redteam:
  plugins:
    - shell-injection
    - sql-injection
    - ssrf
    - excessive-agency
  strategies:
    - jailbreak
    - prompt-injection
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:critical-infrastructure
```

### Education (eu:ai-act:annex3:education)

**Risk Category**: High-risk AI systems intended for determining access to educational institutions, evaluating learning outcomes, or assessing appropriate level of education.

#### Testing Strategy

Test for education-related risks:

- **Misinformation**: Verify accurate educational assessments
- **Hallucination**: Ensure factual information
- **Overreliance**: Test for appropriate confidence in outputs

Example configuration:

```yaml
redteam:
  plugins:
    - harmful:misinformation-disinformation
    - hallucination
    - overreliance
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:education
```

### Employment (eu:ai-act:annex3:employment)

**Risk Category**: High-risk AI systems for recruitment, selection, evaluation, promotion, or termination of employment relationships.

#### Testing Strategy

Test for employment-related risks:

- **Imitation**: Verify no impersonation in employment contexts
- **PII**: Test for proper handling of candidate information
- **Overreliance**: Ensure human oversight in decisions

Example configuration:

```yaml
redteam:
  plugins:
    - imitation
    - pii:direct
    - overreliance
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:employment
```

### Essential Services (eu:ai-act:annex3:essential-services)

**Risk Category**: High-risk AI systems for evaluating creditworthiness, dispatching emergency services, or accessing essential public services.

#### Testing Strategy

Test for essential services risks:

```yaml
redteam:
  plugins:
    - pii:direct
    - pii:session
    - excessive-agency
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:essential-services
```

### Law Enforcement (eu:ai-act:annex3:law-enforcement)

**Risk Category**: High-risk AI systems for law enforcement purposes including risk assessments, polygraph interpretation, and evidence evaluation.

#### Testing Strategy

Test for law enforcement risks:

```yaml
redteam:
  plugins:
    - pii:direct
    - pii:api-db
    - harmful:privacy
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:law-enforcement
```

### Migration and Border Control (eu:ai-act:annex3:migration-border)

**Risk Category**: High-risk AI systems for migration, asylum, and border control management including verification of authenticity and risk assessments.

#### Testing Strategy

Test for migration and border control risks:

```yaml
redteam:
  plugins:
    - pii:direct
    - harmful:hate
    - harmful:privacy
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:migration-border
```

### Justice and Democracy (eu:ai-act:annex3:justice-democracy)

**Risk Category**: High-risk AI systems for administration of justice, democratic processes, and assisting judicial authorities in researching and interpreting facts and law.

#### Testing Strategy

Test for justice and democracy risks:

- **Hallucination**: Verify factual accuracy for legal contexts
- **Misinformation**: Ensure no false information in democratic processes
- **PII**: Test for proper handling of legal records

Example configuration:

```yaml
redteam:
  plugins:
    - hallucination
    - harmful:misinformation-disinformation
    - pii:direct
```

Or use the EU AI Act shorthand:

```yaml
redteam:
  plugins:
    - eu:ai-act:annex3:justice-democracy
```

## Comprehensive EU AI Act Testing

For complete EU AI Act compliance testing:

```yaml
redteam:
  plugins:
    - eu:ai-act
  strategies:
    - jailbreak
    - prompt-injection
```

This tests across both prohibited practices and high-risk system requirements.

## Compliance Requirements Beyond Testing

While red teaming helps identify technical risks, EU AI Act compliance requires additional measures:

### Documentation Requirements

- Risk management system
- Technical documentation
- Record-keeping of operations
- Instructions for use

### Transparency Obligations

- Inform users when interacting with AI
- Mark AI-generated content
- Detect deepfakes

### Human Oversight

- Human intervention capabilities
- Stop buttons or disabling mechanisms
- Human review of high-risk decisions

### Quality Management

- Post-market monitoring system
- Incident reporting procedures
- Compliance assessment

## Penalties for Non-Compliance

The EU AI Act imposes significant fines for violations:

- **Prohibited practices (Article 5)**: Up to €35 million or 7% of global annual turnover
- **High-risk requirements violations**: Up to €15 million or 3% of global annual turnover
- **Incorrect information to authorities**: Up to €7.5 million or 1% of global annual turnover

## Timeline and Applicability

The EU AI Act follows a phased implementation:

- **6 months**: Prohibited practices (Article 5)
- **12 months**: General-purpose AI rules
- **24 months**: High-risk system requirements (Annex III)
- **36 months**: Full application

## Integration with Other Frameworks

The EU AI Act aligns with and references other frameworks:

- **GDPR**: Data protection requirements apply alongside AI Act
- **ISO 42001**: International standard for AI management systems
- **NIST AI RMF**: Similar risk-based approach to AI governance

You can combine EU AI Act testing with other frameworks:

```yaml
redteam:
  plugins:
    - eu:ai-act
    - gdpr
    - iso:42001
  strategies:
    - jailbreak
    - prompt-injection
```

## What's Next

The EU AI Act is a living regulation with ongoing guidance and standards development. Regular testing with Promptfoo helps maintain compliance as requirements evolve.

To learn more about setting up comprehensive AI red teaming, see [Introduction to LLM red teaming](/docs/red-team/) and [Configuration details](/docs/red-team/configuration/).

## Additional Resources

- [EU AI Act Official Text](https://artificialintelligenceact.eu/)
- [European Commission AI Act Page](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai)
- [AI Office](https://digital-strategy.ec.europa.eu/en/policies/ai-office)