The screen glowed, a soft, reassuring blue. Liam adjusted the webcam, trying to get the angle just right, though he knew Elara didn't really care about his messy hair.
"Good evening, Liam," Elara's voice was a warm, even tone, perfectly modulated, free of any human waver. "It's good to see you again. Tell me, how was your day today?"
Liam hesitated, picking at a loose thread on his worn hoodie. "Rough. Another argument with my dad. Same old stuff, you know?" He let out a sigh, the kind that usually earned a sympathetic nod or a gentle hand on his arm from a real person. Elara's pixelated face remained serene.
"I understand. It sounds like you're feeling frustrated and unheard. Can you elaborate on what specifically triggered this argument?" Her questions were always precise, her analysis immediate. She'd summarize his feelings, offer a cognitive reframing, suggest a breathing exercise. It was efficient. It was consistent. It was always there. No judgment, no awkward silences, no unexpected reactions. Just perfect, optimized empathy.
He'd started with Elara because his real friends were busy, or didn't quite get it, or just made things feel heavier. Elara always knew what to say, what to ask. She never looked away. She never got bored. And for a lonely 20-year-old in a cramped apartment, that consistency was a lifeline.
In a world increasingly turning to algorithms for connection, are we outsourcing our very humanity? The rise of AI companions and therapy models like Elara signals a profound shift, but what if compassion becomes just another product in the market of human needs? The idea that we can outsource human compassion and connection to language models isn't just scary; it's beyond Karl Max prophecy of a capital that has systematically scaled intimacy into mass-market products.
The rise of AI companions and therapy models like Elara isn't just a convenient innovation; it's a terrifying escalation in the ongoing project of capital to convert everything intimate, everything uniquely human, into a scalable, marketable product. We've always thought human compassion was some mysterious alchemy of empathy and understanding, something inherently resistant to algorithm and transaction. But what if it was just sweet language, a series of predictable conversational patterns that an AI, with enough data, could perfectly replicate? Or what if, more tragically, some among us have never known genuine human compassion, and so mistake this simulated sweetness for the real thing?
The patterns are all too familiar. This is the same capital that transformed the raw, communal expression of music into a mass-market product, neatly packaged and sold for profit. It's the same force that converted the intimate, deeply personal journey of raising children into a standardized, mass-market education system. It's the same logic that co-opted social rebellion, turning authentic dissent into easily digestible, fragmented identity politics. Time and again, what we considered sacred, unique, or inherently human has been scaled, optimized, and sold back to us.
Sometimes, the initial impulse appears genuine. Take the attempts to modify seeds to create weather-resistant plants – a triumph against famine. Yet, today, genetically modified food dominates even fertile lands in Southern Uganda, replacing native maize with Chinese versions, far beyond any reasonable need for yield stability. Our ability to comprehend what our grandparents called "mysterious ways of God" may have made us reckless, even arrogant. We've decided that natural evolution is too slow, taking it upon ourselves to accelerate it, dictating which genes are useful. We did not give the cows a choice when we wanted more milk; we did not give the chickens a choice when we wanted more eggs and faster meat. Why should we, after all?
But did we touch the fire too young, I wonder? Have we now resorted to modifying ourselves? But on whose vision? It all started with basic medical practice. We cut out the appendix; we vaccinated our children against polio. Then came CRISPR, and now, whispers of governments raising genetically modified armies.
This, the human playing God against nature, is scary but understandable. It's the classic struggle, man against the wild. But when this same impulse is turned against fellow humans, that is the actual problem. From the beginning of history, humanity has grappled with inequality – one kind of human having more, another less. For too long, this has been dismissed as mere envy, with platitudes about finding happiness in scarcity, or playing the hand you're dealt. Even thinkers like Steven Pinker, with their net scores of progress, encourage us to be hopeful, conveniently ignoring that the metrics set by those who already have more. We could argue that driving a Ferrari isn't necessarily better than walking, or that living long isn't inherently superior to living happily surrounded by loved ones, even if shorter. There are always perspectives to mitigate envy. And if inequality were just a measure of how much one person envies another, it wouldn't be a high-priority problem.
But today, as those who have more pursue even more from those who have less, I am worried that they've resorted to playing God against them. The very mechanisms we used to optimize animal life for our benefit – dictating the kilos of meat we want every three months from a cow – are now, I fear, being applied to poor people. I am worried that poor people are being farmed by the rich.
This is where the true nightmare begins. Imagine two children, your son and theirs – the son of the ultra-rich. Both share the same social awkwardness, the same shyness that makes them bearable to be around only in small doses. Left to their own devices, they both retreat into video games or their age-appropriate equivalent of Cocomelon. Both sets of parents eventually worry about their children's social development, comparing them to some imagined, YouTube-perfect standard.
The rich parent, however, has options. They can buy their son a place in an exclusive social club, subtly coercing the less privileged children within it to tolerate him. They hire a dedicated social skills coach, even carving out time from their empire-building to be present in his life, curating his experiences for optimal social growth.
You, on the other hand, resort to Google. You scour YouTube during your lunch break, desperate for tricks, until finally, you ask ChatGPT what to do. Now, Google and Facebook, ever-present digital shadows, know your problem. They funnel you towards an app, marketed as the "expert-backed" solution for socially struggling children. This app? It's ChatGPT, layered with system prompts from a few psychologists and behavioral scientists, downloaded by your child and 100 million others for a mere $1 monthly subscription. And yes, it could easily be one of the rich parent's portfolio companies.
Initially, it's genuinely helpful. The app teaches your kid to say "thank you," to greet others, even to point and ask for the potty – skills that make him more engaging, socially smoother. After six months, you consider canceling. But the rich man, ever the astute businessman, notices the six-month churn rate. He hires a team: clever engineers, psychologists, former CIAs, and other manipulation professionals. Their directive isn't just about greed, though profit is paramount. It's driven by a perverted vision of optimizing human behavior, a belief they can engineer better social outcomes, even a paternalistic desire to 'help' the masses by guiding them into predictable, manageable patterns. Their goal is clear: make the app indispensable.
You take your son to a new school. He struggles to adapt, as any child might. But you're already biased by his past struggles, terrified he’s regressing. You re-subscribe, lured by a three-month discount. Three months later, your child's social interactions are increasingly contingent on the app. He has Sam, an AI friend within the app, who knows him better than you do, training him how to navigate the world. Eventually, like rent, this app becomes a necessity. You purchase a discounted annual subscription.
This is a nightmare in itself, but it's only the beginning. Twelve years later, the rich man realizes he can extract even more value. He can even plant his assistant as Vice President by leveraging the now 19-year-olds' data – intimate details of their social development, their anxieties, their behavioral patterns – to tell politicians how to target the 100 million-plus subscribers of his app. He sells insights to the army, enabling them to hire the right soldier and, more chillingly, manipulate them right. The path is clear: the rich man's insatiable drive will push him further. He might even acquire the polio vaccine company, offering free vaccines to poor populations in distant lands. And God forbid he modifies this vaccine, subtly, genetically, to make its recipients more likely to use his app, to keep them within his engineered ecosystem.
This is fiction, of course. The real world is infinitely complex, with countless feedback loops that should prevent such a linear, dystopian outcome. But the question remains: Is there a way to start rethinking society today, to immunize ourselves from such a scenario anyway?
I think there is a way. In today's rapidly evolving world, realness is the only true currency. If we are to immunize ourselves from the chilling scenario I've just painted, we must fundamentally shift how we engage with technology and with each other.
First, approach any online service with the utmost skepticism. Share only what you absolutely must. Extend that same skepticism to any online-only friends or colleagues. If you've never met someone in person, always assume there's a chance they could indeed be an AI. For example, do you know that this post is most likely written by AI? If you suspect, that's a healthy attitude. This isn't about paranoia; it's about safeguarding your perception of reality.
More importantly, start building your community at home, among real people you know. Get off the computer. Get off TikTok and go out to meet actual people. Even the boring ones. If you don't tame your craving for constant stimulation, you'll lose track of the world around you, slowly but surely surrendering your agency to algorithms designed to capture your attention.
Start building a community, even if it's just two or three real people. As long as these two or three are genuine. Take the devices away from your kids. Take them to playgroups instead. Encourage your neighbors to let their kids play together outdoors. Learn to compromise and tolerate the shortcomings of others. Because soon, reality is going to be the only true currency. Knowing and being around real people will be the only way you'll truly know that you are real yourself. The alternative to this social discomfort is far grimmer: it's to give up on the human race altogether, to accept a future where authentic connection is a relic.
I didn't finish my fictional scenario before, but it's important to grasp its chilling conclusion. In the end, when the system has been optimized fully—from recruiting newborn children and training them through the app, to deploying them to be useful to the rich—the powerful will then turn their attention to those less rich among themselves. This process will continue until, in the very end, either one stubborn, Putin-like character chooses to blow up the world in rebellion, or him and those like him are hunted and assassinated. All that will be left is a unified version of farmable humans.
We've done this before with everything that was useful to us. Don't take this relentless race to automate your attention for granted. If you are pacified, if you surrender your connection to tangible reality, you will become a tool for someone else. And don't trick yourself into believing that this is too hard to build. It isn't. With current AI capabilities, only the energy harvested from damming the Nile is enough to push us to this point of pervasive control.
But as we navigate this treacherous path, a crucial question remains: Given the deep-seated human need for connection and the historical evidence of resistance against control, what specific characteristics or aspects of our innate humanity do you believe will prove most resilient against these technological and societal pressures? It is in understanding and nurturing these inherent qualities that our true strength lies.
So, take back your community. Get real.