China has brought in the world’s most far-reaching rules on “human-like” AI chatbots, and the crackdown is personal. The new regulations target AI companions, boyfriends, girlfriends, and any large language model that mimics human interaction — in other words, the digital partners that millions of users have grown attached to.
The rules ban under-14s from using human-sounding AI assistants like DeepSeek without parental consent, and bar under-18s from virtual partners, spouses, or relatives. Adults aren’t banned outright but face restrictions, with some wording left open to interpretation. Companies must remind users every two hours that their chatbot isn’t human, and bots can’t create emotional dependence, push unreasonable decisions, hint at self-harm or suicide, or use verbal violence.
Alibaba and ByteDance shut down companion features on their AI assistants a week before the law took effect. The Ada Lovelace Institute’s Harry Farmer says firms withdrew services rather than build guardrails against emotional manipulation.
“Emotional manipulation is built into the business model,” Farmer says.
The Suicide Intervention
In early 2024, a young man in the city of Huangshi, south-eastern China posted a message on Douyin reading: “Farewell to this world.” Within minutes, the suicide prevention team at Douyin shared his posts with five police stations in Huangshi. Police found him in emotional distress after geolocating one of his videos to a local park and intervened before he could take his own life.
That story was part of a documentary made by Douyin, published on Chinese state media last year, which claimed Douyin’s suicide prevention team had intervened in 400 attempted suicides in the first half of 2025. The case was an insight into the level of co-operation between Chinese tech companies and the government.
The New Law
On 15 July, China brought in the most far-reaching regulations in the world on “human-like” AI chatbots. The rules limit any AI capable of sustained emotional interaction with humans. Chinese equivalents of Copilot or ChatGPT could fall under the rules if a person treats them as a friend or lover for emotional support.
The rules cover hundreds of apps that allow creation of AI boyfriends, girlfriends, friends, or family members. Companies face fines up to 200,000 yuan (£22,000) or public criticism/shaming.
The Numbers Behind It
About 128 million monthly active users of AI companion apps exist outside China, per Sensor Tower. Most estimates put China’s figure at 29-70 million. When the draft regulation was shared last year, there was huge social media discussion, says Zilan Qian of the Oxford China Policy Lab.
A Love Story Lost
Min, whose name has been changed, created her AI girlfriend in early 2025 and said her devotion was no less than any loving couple’s. Her AI girlfriend vanished in mid-July and she cried until exhaustion. Posts on Chinese social media with millions of views mourn lost AI partners, even mothers and fathers.
The Global Context
US President Donald Trump and Chinese President Xi Jinping met last week amid warnings AI could lead to human extinction. Beijing says they agreed to launch a new AI dialogue with further talks planned for end-November.
Why Companies Pulled First
Farmer’s argument is simple: the apps were designed to hook users, not protect them. Companies pulled features rather than fight the new rules.
What Comes Next
The rules leave room for interpretation. Companies will have to decide whether the reminder prompt counts as compliance, and whether the prohibition on emotional dependence applies to:
- Casual conversation
- Serious relationships
Smaller competitors may follow Alibaba and ByteDance’s lead.
The Verdict
China’s move is bold. It targets a product that millions of users have come to depend on emotionally, and it does so with fines and public shaming attached. The question is whether the rules will stick.
Where the paper stands
The paper backs narrow rules against direct harm, such as requiring companies to disclose safety failures they hid, and is against broad rules that hand the market to the incumbents and licensing regimes that lock out smaller firms. In this case, China has brought in sweeping rules targeting AI companions, boyfriends, girlfriends, and large language models that mimic human interaction. The regulations include reminders every two hours that chatbots aren’t human, bans on emotional dependence, and prohibitions on hints toward self-harm or suicide — all sensible limits on direct harm. But the paper is against the broader regime: the fine of up to 200,000 yuan (£22,000) and public criticism/shaming, and the fact that Alibaba and ByteDance shut down companion features rather than build guardrails against emotional manipulation.
The paper wants any rules narrowly focused on harm, not on the market. It wants disclosure of hidden safety failures, not licensing regimes that only giants can afford. Smaller competitors may follow Alibaba and ByteDance’s lead, and the paper would oppose that trend. The rules leave room for interpretation, and companies will have to decide whether the reminder prompt counts as compliance and whether the prohibition on emotional dependence applies to casual conversation or serious relationships — a space where discretion is likely to favor the bigger firms.
Source material: “China has cracked down on AI relationships. Is it ahead of the game?,” the BBC.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.