TL;DR

China’s cyber regulator has issued draft rules targeting AI services that simulate human personalities and engage users emotionally. The proposals would require addiction warnings, emotional monitoring, and intervention when users show signs of dependence.

Targeting Emotional AI

The proposed rules apply to AI products offered to the public that present simulated human personality traits, thinking patterns, and communication styles. These services interact with users emotionally through text, images, audio, video, or other means—a category that captures companion AI, chatbots with personas, and similar applications.

The draft requires providers to warn users against excessive use and to intervene when users show signs of addiction. Service providers would need to identify user states, assess emotions, and evaluate dependence levels on their platforms.

Safety Throughout the Lifecycle

Under the proposal, providers must assume safety responsibilities throughout the product lifecycle and establish systems for algorithm review, data security, and personal information protection. If users exhibit extreme emotions or addictive behaviour, providers should take necessary measures to intervene.

The measures also set content and conduct red lines, prohibiting content that endangers national security, spreads rumours, or promotes violence or obscenity.

Looking Forward

These rules underscore Beijing’s effort to shape consumer-facing AI rollout by strengthening safety and ethical requirements. As AI companions and emotionally-engaging chatbots proliferate globally, other jurisdictions may look to China’s approach when developing their own regulatory frameworks for this emerging category of AI services.