Incident 508: Celebrities' Deepfake Voices Abused with Malicious Intent

Responded
Description: Voices of celebrities and public figures were deepfaked using voice synthesis for malicious intents such as impersonation or defamation, and were shared on social platforms such as 4chan and Reddit.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: ElevenLabs developed an AI system deployed by Reddit users , ElevenLabs users and 4chan users, which harmed public figures and celebrities.

Incident Stats

Incident ID
508
Report Count
4
Incident Date
2023-01-30
Editors
Khoa Lam
Applied Taxonomies
MIT

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

4.3. Fraud, scams, and targeted manipulation

Risk Domain

  1. Malicious Actors & Misuse

Entity

Human

Timing

Post-deployment

Intent

Intentional

AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse
vice.com · 2023

4chan members used ElevenLabs to make deepfake voices of Emma Watson, Joe Rogan, and others saying racist, transphobic, and violent things.

It was only a matter of time before the wave of artificial intelligence-generated voice startups bec…

Tweet: @elevenlabsio
twitter.com · 2023
ElevenLabs post-incident response

Crazy weekend - thank you to everyone for trying out our Beta platform. While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to Twitter comm…

Emma Watson reads Mein Kampf on 4Chan in deepfake audio trick
thetimes.co.uk · 2023

A British start-up has been forced to introduce safeguards after deepfake audio recordings of the actress Emma Watson reading Mein Kampf by Adolf Hitler and the broadcaster Sir David Attenborough being racist were released.

Users took the v…

Deepfake audio: Emma Watson's case is proof voice-cloning is a problem set to rise
cosmopolitan.com · 2023

Since finding fame at the tender age of ten, through her role as Hermione Grainger in the Harry Potter movie franchise, Emma Watson has become a household name. She's also, sadly, become one of the most deepfaked celebrities online.

This wo…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Previous IncidentNext Incident