Combat Misleading Deepfake Endorsements By Changing Enforcement Approach
5 de junio de 2025
The Board has overturned Meta’s decision to leave up a Facebook post showing an AI-manipulated video of a person who appears to be Brazilian soccer legend Ronaldo Nazário endorsing an online game. Taking the post down is consistent with Meta’s Community Standards on fraud and spam. Meta should also have rejected the content for advertisement, as its rules prohibit using the image of a famous person to bait people into engaging with an ad.
Based on public reporting, the Board notes Meta is likely allowing significant amounts of scam content on its platforms to avoid potentially overenforcing a small subset of genuine celebrity endorsements. At-scale reviewers are not empowered to enforce this prohibition on content that establishes a fake persona or pretends to be a famous person in order to scam or defraud. Meta should enforce this prohibition at-scale by providing reviewers with often easily identifiable indicators that distinguish AI content.
About the Case
In September 2024, a user shared a post involving an AI-manipulated video of a person who appears to be retired Brazilian soccer player Ronaldo Nazário. In the video, he encourages people to download an app to play the popular online game Plinko (or Plinco).
In the video, the audio imitating Ronaldo Nazário is not in sync with his lip movements. The video also shows unrealistic AI-generated images of a schoolteacher, a bus driver and a grocery store worker, and the average salary for their jobs in Brazil. The audio claims that players on Plinko can earn more money from the game than the jobs mentioned. The video encourages users to click a download link to the app, although this leads to a different game called Bubble Shooter. The post was viewed over 600,000 times.
A user reported the content to Meta as a fraud or scam, but the report was not prioritized. The company did not remove the content. The user appealed this decision to Meta, but this appeal was not prioritized for human review either, so the content remained on Facebook. Finally, the user appealed Meta’s decision to the Board, saying the post appeared to be sponsored. If a post is boosted, an ad is created on the post.
The ad was disabled for violating the company’s Unacceptable Business Practices Advertising Standard, although the original organic post remained on the platform. After the Board identified this case for review, Meta removed the original post for violating the Fraud, Scams and Deceptive Practices policy. Meta later confirmed the post also violated its Spam policy.
Deepfakes and deepfake endorsements are increasing globally, including those involving public figures promoting fraudulent political campaigns and financial scams. Reports highlight that many of the financial scams in Brazil that originate on Facebook, Instagram and WhatsApp involve AI-manipulated content.
Key Findings
Removing the content is consistent with Meta’s human rights responsibilities. Misleading manipulated endorsements pose significant risks to the depicted person’s rights to privacy and reputation. They also impact the public, by potentially facilitating fraud.
The Board is concerned that at-scale content reviewers are unable to remove posts that establish a fake persona or pretend to be a famous person “in an attempt to scam or defraud,” even if the content contains clear indicators that it violates Meta’s policies. Such content can only be removed by Meta’s specialized teams, making underenforcement of its Fraud, Scams and Deceptive Practices policy more likely.
Meta is likely allowing significant amounts of scam content on its platforms to avoid potentially overenforcing a small subset of genuine celebrity endorsements. This is particularly concerning when genuine celebrity endorsements will likely have other protections against overenforcement, either through formal systems such as cross-check or points of contact at Meta. The Board therefore recommends Meta change its approach and enforce this policy line at-scale.
The manipulated or fake nature of the video is apparent. The Board finds the post violates Meta’s prohibition on fake personas or pretending to be a famous person to scam or defraud, under the Fraud, Scams and Deceptive Practices Community Standard. It also violates Meta’s prohibition on sharing deceptive or misleading links under its Spam Community Standard, as it promotes Plinko but links to a different game. Therefore, the Board finds that the post should have been removed when reported. Even before the post’s removal, Meta should have applied an “AI info” label, under its Manipulated Media policy. Meta should also have rejected the content for advertisement, as its Unacceptable Business Practices Advertising Standard prohibits using the image of a famous person and misleading tactics to bait people into engaging with an ad.
Meta has a responsibility to “mitigate adverse human rights impacts” of monetized content that could scam or defraud – in line with the United Nations Guiding Principles on Business and Human Rights. When paid to boost content, Meta should ensure these posts do not violate its policies.
The Oversight Board’s Decision
The Oversight Board overturns Meta’s decision to leave up the post on Facebook.
The Board also recommends that Meta:
- Enforce at scale its Fraud, Scams and Deceptive Practices policy prohibition on content that “attempts to establish a fake persona or to pretend to be a famous person in an attempt to scam or defraud” by providing reviewers with indicators to identify this content. This could include, for example, the presence of media manipulation watermarks and metadata, or clear factors such as video-audio mismatch.
Further Information
To read public comments for this case, click here.
