Description: A coordinated neo-Nazi network on TikTok used AI-generated media, including Hitler speeches, to spread Nazi propaganda and extremist content, violating TikTok’s hate speech policies. The network evaded platform moderation through coded language, imagery, and music, with some accounts accumulating millions of views. TikTok’s algorithm further amplified the reach of this content, despite community guidelines.
Editor Notes: Reconstructing the timeline of events: On February 6, 2023, an account with violent antisemitic content calling for armed revolution began posting on TikTok. In May 2024, the Institute for Strategic Dialogue (ISD) created two dummy accounts to track how TikTok's algorithm recommended pro-Nazi content. By June 3, 2024, ISD had identified a Nazi account that had accumulated over 87,000 views before being banned. Despite this, on June 6, 2024, a heavily extremist pro-Nazi account remained active, recruiting members to off-platform groups. Finally, on July 29, 2024, ISD published its full report, revealing a coordinated network of over 200 accounts promoting Nazism and extremist content, some of which used AI-generated media to bypass TikTok’s moderation efforts. For this incident ID, I am taking the ISD's date of publication for their findings as the incident date.
Note on "NazTok": The ISD report uses the term "NazTok" in their title for the report as a shorthand for "Nazi TikTok" while not explicitly defining their use of this term beyond that inferred meaning. I am replicating "NazTok" in the Deployers field to tie likely new incidents that will emerge together in this particular genre of incident. I have retroactively applied "NazTok" to Incident 809's Deployer field too.
Entités
Voir toutes les entitésPrésumé : un système d'IA développé par Pro-Nazi TikTok accounts et mis en œuvre par Unknown AI developers, endommagé TikTok users.
Statistiques d'incidents
ID
810
Nombre de rapports
1
Date de l'incident
2024-07-29
Editeurs
Daniel Atherton
Rapports d'incidents
Chronologie du rapport
isdglobal.org · 2024
- Afficher le rapport d'origine à sa source
- Voir le rapport sur l'Archive d'Internet
translated-fr-Self-identified Nazis are openly promoting hate speech and real-world recruitment on TikTok. Not only is the platform failing to remove these videos and accounts, but its algorithm is amplifying their reach.
Content warning: T…
Variantes
Une "Variante" est un incident qui partage les mêmes facteurs de causalité, produit des dommages similaires et implique les mêmes systèmes intelligents qu'un incident d'IA connu. Plutôt que d'indexer les variantes comme des incidents entièrement distincts, nous listons les variations d'incidents sous le premier incident similaire soumis à la base de données. Contrairement aux autres types de soumission à la base de données des incidents, les variantes ne sont pas tenues d'avoir des rapports en preuve externes à la base de données des incidents. En savoir plus sur le document de recherche.
Incidents similaires
Selected by our editors
Did our AI mess up? Flag the unrelated incidents
Amazon Censors Gay Books
· 24 rapports
Incidents similaires
Selected by our editors
Did our AI mess up? Flag the unrelated incidents
Amazon Censors Gay Books
· 24 rapports