Meta’s Latest Approach Helps Deliver Content Moderation At Scale, But Is Not Without Risk
By Evelyn Aswad, Paolo Carozza, Pamela San Martin and Helle Thorning-Schmidt, Oversight Board Co-Chairs
Late last week, Meta announced plans to use more advanced AI models in its content moderation and user support systems, asserting that the AI models will drive down scams, find and prevent accounts that are impersonating celebrities, catch more violating content, and can do these things in 98% of the languages spoken by people online.
If its proposals live up to their goals, they have the potential of driving significant benefits, including on things the Board has emphasized for years, such as protecting against false positives that unnecessarily remove speech, providing better explanation to users, detecting and enforcing violating content at a scale human review alone cannot reach, sparing human reviewers from some of the most odious and harmful content, and improving moderation in low-resource languages.
Although promising, Meta’s move toward more advanced AI models is not without risk. It represents a significant shift that requires an increase in data gathering and assessment, transparency on the findings of those assessments, and independent oversight on how decisions are made throughout this critical period of transformation.
We know from experience that AI systems remain imperfect; they still struggle with the nuances of sarcasm, humor and coded language. Meta must also be vigilant in guarding against inherent biases and hallucinations, and ensure mitigations can keep pace with fast-breaking global crises. In high-stakes environments where human rights are on the line, failing to account for such shortcomings can have disastrous consequences.
We also have concerns about LLMs effectively deciding what speech should stay on platforms and what cannot, without human rights considerations necessarily at their core. Technology companies should be clear in how they are aligning LLMs to human rights standards, and also run regular audits based on actual performance to allow for “human-in-the-loop” retraining.
The Board strongly believes that independent, transparent oversight is necessary regardless of whether content moderation is conducted by people or by artificial intelligence.
As companies rapidly evolve to expand artificial intelligence, the challenges and concerns surrounding these technologies are also increasing. The Board strongly believes that independent, transparent oversight is necessary regardless of whether content moderation is conducted by people or by artificial intelligence. In addition, as so much is still unknown on the limitations and risks of these models, we believe ongoing continuous review will be essential to test companies’ systems in a variety of contexts.
Finally, transparency is essential for trust. We urge Meta to practice maximum transparency and share the results of its testing and red-teaming with the broader public to demonstrate how these models perform across different cultures and conflict zones.
This is an important moment for Meta, which it must use to remedy the lack of transparency on content decisions that has frustrated users for so long and raised concerns on freedom of expression. The Oversight Board stands ready to engage in independently assessing Meta’s evolving systems and evaluating their alignment against the human rights principles that are at the core of the Board’s mission and function.
Expect to hear more from the Board about these changes in the weeks and months ahead.
