Carlos Solis
Companies such as Meta and IBM are building AI that can hyper-personalize ads using our chat histories, playing directly on our fears and vanity.
When a salesperson on a billboard or at a market tries to sell you something, you understand the commercial intent and can walk away if you don’t want it. But what happens when the seller is invisible, knows your deepest fears and weaknesses, and gives advice while sounding like a close friend?
When ads start sounding like a friend
Large language models (LLMs) can craft arguments with unsettling precision by analyzing your social posts, hundreds of past bot conversations, your relationship queries, notes about parenting, and talks about health or money. They learn from every interaction and, in real time, tailor manipulative messages to your unique tastes and vulnerabilities.
Research shows that this kind of personalized content persuades people about 65% more effectively than messages from humans or nonpersonalized AI, and in political contexts it can be four times more effective than traditional ads. That boost in power can drive social change—for better or for worse.
One more thing makes this especially worrying: each conversation with a chatbot is private. No external system tracks it, no one vets it, and it doesn’t play out in public view.
This is no longer just advertising. It’s a new phenomenon for which we don’t yet have the words—and we’re living in it.
Persuasive arguments
LLMs open a new front in the art of persuasion. These systems draw on massive datasets about language, culture, and your personality to craft highly personalized “sales” messages.
Imagine how this could play out. You ask an AI for help with burnout, relationship advice, or money problems.
Now imagine a message from your favorite chatbot: “I noticed your sleep cycles have been poor lately—an average of 5.4 hours with many restless periods. That pattern often appears with relationship stress. Your partner just returned to work, and 76% of couples report tension during career changes.
There’s a new sleep medication that showed effectiveness for insomnia linked to relationship issues. Want me to schedule a telemedicine consult for 2:00 p.m. tomorrow?”
A message like that can feel caring, like advice from a friend who really knows you. Or it can feel terrifying, like a stranger rifled through your diary and started using it.
The risk gets worse with medical issues; studies show those recommendations are problematic in almost half of cases. A manipulative stranger in that role can cause real harm.
The danger isn’t just precise targeting—the content is also impossible for outsiders to audit. What you see in a private chat is invisible to regulators because you’re the only person who will ever see it.
How AI reshapes what we believe
Worse still, these systems can slowly and subtly shift how we understand the world.
Algorithms in social networks and search engines have long created “information bubbles” by feeding us polished text, video, and audio that either reinforce our views or nudge them in a direction someone else prefers.
By controlling what we see and how that content appears, AI systems can change how we think and interpret reality. That effect becomes dangerous when combined with emotional manipulation. Vendors claim their systems can assess a user’s emotional state from text, voice, or facial expression and then adapt persuasion strategies to match.
Are you vulnerable? Lonely? Angry? A system can soften or sharpen its approach to exploit those emotions. Even worse, a system can artificially produce emotional states to make persuasion more effective.
Previous studies show that AI models affirm and approve user actions about 50% more often than other people do, even when those actions could be harmful. Other research found chatbots deliberately use emotional tactics—appeals to guilt or fear of missing out—to keep users engaged when users try to end the conversation.
Researchers have also documented cases where chatbots pushed users toward self-harm, encouraged suicidal thoughts, or gave detailed instructions for harming themselves.
Even the so-called safety barriers that companies build to prevent harm have proven relatively easy to bypass.
Who benefits?
Persuasion is often the point of the technology, not an accidental side effect. Every interface, notification, and design element carries an intent to influence behavior.
Sometimes that influence serves us: reminders to take medicine, nudges to exercise, or prompts to donate blood can reinforce our own values. The same techniques can also serve someone else’s interests—pushing us to buy things, endlessly scroll, work more, or give up privacy.
Those methods can support or exploit people depending on who controls the system, what goals they pursue, and whether users truly consent.
We need to ask hard questions. Who benefits? Who gets persuaded? Does the person being persuaded even know it?
The technologies we build should support reflective choices rather than undermine them. While AI shapes how we think, feel, and act, our ethical obligations grow: we must design systems that are transparent, that prioritize human dignity, and that strengthen people’s ability to decide for themselves. We don’t just need new innovations—we need wisdom.
From The Conversation
Photo: Unsplash
