A recent news conference in Washington, D.C., announced the Take It Down Act, aimed at tackling non-consensual sexual content online.
According to a report by the nonprofit Tech Transparency Project (TTP), Meta Platforms Inc. has violated its own advertising policies by allowing thousands of ads for AI-driven "nudify" applications on Facebook and Instagram. These ads have been primarily disseminated by GatherOne Inc., a Beijing-based advertising partner that facilitates access for Chinese companies seeking visibility on Meta’s platforms. The TTP findings indicate that over 7,600 ads for mobile applications capable of swapping women's faces onto naked bodies—and even producing sexualized content—have been run recently. One particular app flagged in the report has been criticized for enabling the sharing of material categorized by some users as child pornography.
Meta’s advertising policies explicitly forbid sexually suggestive content, including ads for apps that strip images of individuals. Yet, TTP's investigations have revealed that major tech firms, including Apple Inc. and Alphabet Inc., have struggled to effectively eliminate access to such problematic applications. This has led to calls for increased accountability from regulators, victims, and their families regarding the proliferation of deepfake pornography.
Katie Paul, director of TTP, remarked, “Meta seems to be turning a blind eye to the violations of one of its main advertising partners in China concerning nudification ads.” Although Instagram and Facebook are not available in China, they remain appealing platforms for Chinese enterprises targeting global markets. Paul interpreted the findings as an indication that Meta may prioritize advertising revenue over user safety.
A representative from Meta, Cindy Southworth, emphasized the company’s stance on banning non-consensual intimate imagery and nudification apps, elaborating that advertisers found in violation of these policies may face penalties, including ad cancellations and account terminations.
GatherOne has responded to inquiries by stating that it adheres to Meta's advertising policies and all relevant international regulations. The company claims to uphold a strict no-tolerance approach towards deceptive practices, explicit content generated without consent, child exploitation, and other fraudulent activities. It supports a variety of industries, including gaming, e-commerce, and finance, by helping them purchase ad placement across platforms like Meta, Google, TikTok, and X (formerly Twitter).
In light of the controversy, GatherOne announced it would halt the creation of new advertising accounts related to AI nudification and face-swapping services while simultaneously implementing compliance upgrades to adapt to the evolving landscape of synthetic media.
The Tech Transparency Project operates as a research entity under the Campaign for Accountability, a nonpartisan organization that monitors corporate and government behavior.
The report is set to be released on Monday and sheds light on how Meta generates revenue from a market of 1.4 billion people where its services remain inaccessible. Chinese businesses aiming to sell products or services internationally often rely on intermediary firms like GatherOne to navigate advertising on global platforms.
For the 2024 fiscal year, Meta reported $18.4 billion in sales from China, making up approximately 11% of its overall revenue and more than doubling its figures from 2022. However, the company has not provided a breakdown of revenue specifics following this.
Sarah Wynn-Williams, a former Meta executive and whistleblower, testified to Congress last year about the tools Meta has allegedly provided to China over the years, enabling the country to advance its position in the global AI arena. She pointed out that China remains a critical market for Meta, evidenced by its ongoing revenue streams from the region. Wynn-Williams is set to share her experiences in her upcoming memoir, "Careless People."
In December, Reuters reported that Meta's internal documents suggested that approximately 19% of its China revenues stemmed from ads promoting prohibited content, such as pornography and scams, with GatherOne identified as an agency involved in facilitating such ads. Meta denied the accuracy of this report but reaffirmed its commitment to combating fraud and illicit activities on its platforms.
Between April and June, TTP discovered and documented numerous ads that Meta ultimately removed after they were shown to users. These ads often displayed suggestive images of young women alongside text claiming, "Create a video with any face."
Notably, GatherOne is listed on Meta’s business website in China, serving as one of 11 approved agents for advertising on its foreign networks. Their website even features a photograph showing their CEO receiving an award alongside Meta executives at a January event in Hong Kong.
In the previous year, Meta indicated it was undertaking measures to crack down on deepfake nudification applications and initiated a lawsuit against a Hong Kong-based developer involved in such services.
TTP’s findings also revealed a particular app, BAfter, which provided users with pornographic face-swapping options. The app was reportedly available on the Google Play store, where reviewers flagged it for containing illegal child pornography. The app was subsequently removed from the platform, and GatherOne stated that it has cut off all advertising access for the associated parties.
In response to inquiries from Bloomberg, Meta confirmed that it had banned BAfter and disabled links to several other nudifying apps. Google has not provided comments regarding the situation.
A recent survey conducted by George Mason University found that over half of 557 teenagers in the U.S. had utilized nudification tools to create images of themselves or others, with a significant number reporting experiences as victims of such tools. According to Chad M.S. Steel, a researcher in digital forensics at George Mason, many teenagers are already familiar with accessing these services, suggesting that actions taken by social media platforms or app stores may have limited effectiveness in curbing their availability.
The Take It Down Act, signed into law by President Donald Trump last year, criminalizes the publication of non-consensual sexual content and mandates social media operators to remove such posts. The Federal Trade Commission began enforcing this law in May.
TTP discovered the connection between the ads and GatherOne by analyzing disclosures in Meta’s ad library. While the extent of app installations driven by these ads remains unclear, initial data indicates that a few of the apps studied have collectively amassed over 2.5 million downloads worldwide, according to market analysis firm AppMagic, as reported by TTP.
