On October 24, 2026, an academic paper titled “Harm Is Behavioural, Moderation Is Lexical” was published on the arXiv platform (ID 2610.31047v1), systematically exposing a fundamental flaw in mainstream automated content-safety systems: they rely excessively on vocabulary and surface-level forms rather than the actual behavioral intent of messages, leading to severe inequities across linguistic communities and cultural expressions.
The study found several key biases through empirical testing: automatic classifiers label African American English (AAE) as “offensive” at a rate up to twice that of other English varieties; some systems assign “toxicity” scores to well-known drag queen social media accounts that are even higher than those of white supremacist accounts; and safety policies that perform well in English virtually fail in low-resource languages such as Arabic and Romanized Hindi—where system accuracy approaches the level of random guessing.
The paper identifies seven categories of failure modes and proposes testable hypotheses: what truly exhibits cross-linguistic robustness is not “harmful vocabulary” but rather harmful behaviors themselves (such as demanding confidentiality, soliciting money, or attempting to isolate others). The research team devised a new evaluation protocol and tested it across five open-source systems—each equipped with keyword dictionaries and two types of multilingual safeguarding models—in eleven languages. The results show that, among 55 system–language combinations, 51 failed to achieve native-speaker-level accuracy in distinguishing between “harmful functions” and “harmless forms.” Moreover, the stronger the model’s capabilities, the more pronounced discrimination became across nine languages; meanwhile, Arabic and romanized Hindi remained at near‑random levels, leading to a widening gap in service quality across languages as technology advanced.
The study also points to a dangerous paradox: a given system exhibits both severe over‑labeling (false positives) and serious under‑protection (false negatives) for the same language. Even more concerning, if only the false positive rate is reported, the worst‑performing language may mistakenly appear to be “handled most fairly”—thereby masking the true inequities in risk allocation. The paper emphasizes that only by shifting to a behavior‑centric evaluation paradigm can we restore the credibility and inclusivity of content moderation systems.