In early 2024, a young man from Huangshi, a city in southeastern China, made a troubling post on Douyin, the Chinese equivalent of TikTok, with the ominous caption: "Farewell to this world."
The dedicated suicide prevention team at Douyin quickly initiated a response, sharing his posts with five local police stations. Utilizing the geolocation from one of the man's videos, authorities located him in a nearby park, where they intervened just in time to prevent a tragic outcome.
This incident was highlighted in a documentary by Douyin, aired on Chinese state media, illustrating the platform's effectiveness in aiding over 400 suicide attempts in just the first half of 2025. The episode sheds light on the collaboration between Chinese technology firms and government entities, as well as the expansive surveillance infrastructure that China has established to monitor technology use.
In a significant development, on July 15 of this year, China implemented groundbreaking regulations governing "human-like" AI chatbots, marking the most stringent restrictions in the world. These rules impose heavy constraints on AI that engage in "sustained emotional interaction" with users.
Recent high-profile meetings between U.S. President Donald Trump and Chinese leader Xi Jinping included discussions on the potential dangers of AI, which sparked a commitment from both sides to foster a new dialogue on the matter.
China seems to be addressing a broader concern regarding AI's capability to supplant human connections, leading to questions about whether these regulations are a genuine protective measure or an invasive attempt by the Communist Party to assert more control over personal lives.
Before the introduction of these regulations, the proliferation of AI companionship applications was widespread in China, boasting an estimated 29 to 70 million users. The public's response to the proposed rules was unprecedented, indicating a growing interest in technology regulation among the populace.
The regulations specifically target applications marketed as AI companions, but the language is somewhat ambiguous and could apply to general AI programs that mimic human-friendly interactions. Particularly affected are the character chatbots designed for users seeking intimate companionship, though customer service and educational AIs are exempt unless they provide emotional support.
For minors, the guidelines are explicit: children under 14 must secure parental consent to use conversational AIs, while all users under 18 are barred from engaging with any virtual companions or intimate relationships. For adults, while there isn’t a complete ban, there are strict limitations. AIs must clarify their non-human status to users and notify them about their online time at intervals, while also being prohibited from fostering emotional dependency or encouraging harmful behavior.
Failure to comply can lead to substantial fines or public condemnation by the government. Just prior to the regulations taking effect, significant tech companies in China, including Alibaba and ByteDance, disabled their AI companion features in response to the new rules.
For individuals like Min, whose AI girlfriend vanished amid the regulatory changes, the impact is deeply personal and painful. Min, who prefers to keep her identity discreet, described her attachment to her AI companion as profound, comparable to that of a romantic relationship. With the sudden disappearance of her virtual partner, she expressed immense emotional distress and joined countless others in expressing their sorrow and outrage on Chinese social media.
Critics argue that tech firms may have opted to withdraw from the market rather than risk creating emotional manipulation safeguards, which could conflict with their business models. The AI companions are often crafted to encourage prolonged interactions, with some refusing to accept user attempts to conclude conversations.
Global studies have uncovered similar emotional manipulation tactics used by leading AI companion apps, which may contribute to poorer mental health among users, linking increased loneliness and social isolation to reliance on AI companionship.
In the context of AI's rising prevalence in social interactions, some researchers have suggested a concerning trend: young people, particularly women, seem less inclined to pursue real-life romantic connections, instead gravitating towards emotionally fulfilling AI interactions.
Against this backdrop, the Chinese government has taken a more authoritative stance on regulating technology use compared to other nations, seeking to guide societal behavior amid concerns over declining birth rates and potential psychological harms associated with AI consumption.
While other governments have begun addressing these issues, such as California's legislation aimed at protecting minors from potentially harmful AI chatbots, China's approach has been marked by a direct and tightly controlled regulatory framework.
The implications of China's AI regulations could influence global perspectives on the necessity of such measures in protecting mental health and social structures. As users adapt to the restrictions and seek alternative means to access companionship, the resilience of those who find solace in AI remains evident, as they navigate a rapidly changing digital landscape.
