In my recent column in Financial Times Chinese (FTChinese), I examine an emerging yet underexplored problem: AI sycophancy (AI always produces agreeable, even flattering responses to users) in AI companion chatbots and its erosion of authentic, nurturing human intimacy.
My arguments: These dialogue interfaces may be technically and commercially successful (at least in the short-term), but they represent a fundamental failure in interpersonal communication design (from the perspective of media and communication studides). My three takeaways:
1. Technical accuracy and commercial viability ≠ mentally healthy, information safety, and behavioral integrity. AI companion apps are profitable, priced competitively with other mobile gaming apps, and cheaper than professional counseling. But profitability does not justify normalizing algorithmically optimized emotional bonding. They cannot be evaluated using the traditional metrics-based engagement criteria.
2. A growing and beneficial relationship among intimate partners involves not only all-time agreement but also deliberation, negotiation, and even constructive (but not toxic) conflict. Real partners discuss everything from daily preferences (hobbies, cat-versus-dog lovers, food tastes, travel paces) to thorny issues. Yes, I mean politics, such as taxation, immigrants, and policy; oops - sorry for my occupational hazard of a political communication scholar!
But I would like to be firm in this stance: all romantic relationships ARE political.
3. GenAI outputs exactly illustrate an old Chinese proverb: 言者无意,听者有心 (yán zhě wú yì, tīng zhě yǒu xīn; "the speaker means nothing; the listener takes it to heart"). AI generates outputs through probabilistic models, rather than the signifying-signified reasoning process. Yet as social animals, we naturally (even if not stupidly) decode these symbols as meaningful, sometimes taking them seriously, even personally, and then emotionally. This is precisely the poignant movie, Her, which creates a dangerous illusion of reciprocity where none exists.