Incident 419: Facebook's Automated Moderation Allowed Ads Threatening Election Workers to be Posted

Description: Facebook's automated moderating system failed to flag and allowed ads containing explicit violent language against election workers to be published.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: Facebook developed and deployed an AI system, which harmed Facebook users.

Incident Stats

Incident ID
419
Report Count
3
Incident Date
2022-12-01
Editors
Khoa Lam
Applied Taxonomies
MIT

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

7.3. Lack of capability or robustness

Risk Domain

  1. AI system safety, failures, and limitations

Entity

AI

Timing

Post-deployment

Intent

Unintentional

Facebook Failed to Stop Ads Threatening Election Workers
nytimes.com · 2022

Facebook says it does not allow content that threatens serious violence. But when researchers submitted ads threatening to "lynch," "murder" and "execute" election workers around Election Day this year, the company's largely automated moder…

“We’re going to kill you all”: Death threats against US election workers approved on Facebook
globalwitness.org · 2022

An investigation by Global Witness and the NYU Cybersecurity for Democracy (C4D) team looked at Facebook, TikTok, and YouTube's ability to detect and remove death threats against election workers in the run up to the US midterm elections.

T…

Facebook failed to detect death threats against election workers ahead of US midterm elections
globalwitness.org · 2022

Facebook was unable to detect three quarters of test ads explicitly calling for violence against and killing of US election workers ahead of the heavily contested midterm elections earlier this month, according to a new investigation by Glo…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Previous IncidentNext Incident