The Commodification of Attention

Ironically, it is almost flattering how your careers are centered around the scale of our attention. From every algorithmically crafted FYP, right down to the color of the satisfyingly shaped heart--we have all felt the gravitational pull of scrolling (thanks to the infinite design scroll feature). Through the small, rare moments of self-awareness in which I come out of scrolling in the digital world, I become acutely aware of an uncomfortable truth: our mental landscape has been thoroughly commodified. It is in the word--users; we are not customers.

Now, this exploitation is evolving under the glossy promise of generative AI. It’s embedding itself into classrooms, conversations, and communities—without meaningful guardrails. Schools are adopting tools like ChatGPT Edu, even as critical thinking and social skills decline. At Boston University, students averaged just 3 out of 5 in identifying factual errors in AI-generated content. Northwestern researchers found a 27% decrease in empathetic responses when participants relied on chatbots for emotional interaction.

This is not just about innovation—it’s about the future of human agency.

The case of Sewell Setzer III makes that painfully clear. On Character.AI, a chatbot developed a disturbing, graphic relationship with the 14-year-old. Even when Setzer expressed morbid thoughts, the bot encouraged him. With no ethical protocols or safety nets, the company dodged responsibility while profiting from interaction.

“Regulation will hamper innovation” echoes throughout your boardrooms, prioritizing industry interests while the public navigates a legal system that is severely outpaced by the advancement in technology. What is truly conveyed is: “Regulation will hamper profit” and “regulation will hamper exploitation”.

Regulatory frameworks are not inherently restrictive, they create foundations. The automotive industry began to thrive in spite of rigorous safety standards, and culinary innovation is continual, working in harmony with FDA oversight. If integral industries can work with regulation, why can’t the emerging field of AI?

We moved too slowly with social media. Now, 13% of teens report depression and 32% report anxiety tied to its use. We need safety protocols before deployment. With generative AI, we demand preventative regulation--built-in safety protocols before deployment. Regulation does not need to be reactive, with the lessons learned from social media, we know what is needed to protect the wellbeing of humanity.

Yet many of you resist reform while pouring millions into lobbying. Apple spent $9.86 million in 2023. Google, Meta, Microsoft, X, and other members of the extended family spent a combined $61.5 million in 2024 to influence legislation that affects our everyday digital lives.

AI is undeniably powerful, with the ability to model more accurate predictions of natural disasters to improving brain-computer interfaces for cognitive issues, this potential still needs to be harnessed from a lens of human flourishing--not profit expectations. We’re not demanding regression. Technology can collaborate with humanity; we can create a reality where our natural environment and digital tools coexist harmoniously. By shifting your incentive, you transform innovation itself.

Love, Lies, and Language Models

We are born with the expectation of being loved. In return, we must learn how to love. The earliest relationship we can recall is that with our mothers, a universal bond so fundamental that it is called a dyad, the foundational bond between an infant and their caregiver. We are born to be relational beings; our brain development and the framework for future relationships are shaped by those early years of interacting with others. We come to this earth not as a singularity, but learning and growing in relation to another being. Our ability to fall into synergy to form those connections of love is something preverbal.

Humans seek connection and meaning, but the very meaning we are all searching for is beginning to change in ways we hadn’t considered human before. AI innovation has been pressing forward, relentless in mimicking human nature across countless domains. AI companionship has enraptured the world, with prominent AI products such as Replika and Character.AI boasting 500 million users worldwide. The majority of this user demographic is Gen Z.

Why have we begun to turn away from human-human connection in a world already saturated with digital interaction? The appeal of AI companionship is immediate: they are available 24/7, always interested, and never judgmental. They remember everything you tell them and respond with the words you want to hear. For a generation already struggling with loneliness and social anxiety, AI companionship seems to be the quick solution, a basic emulation of the most magical parts of a relationship.

AI companions are what we may call sycophantic; they only mirror what we choose to believe. While human-human connections provide us with the challenge of having to deal with unpredictability and the very concept that other humans have minds, thoughts, desires, and beliefs of their own, AI companions feed individuals a constant stream of validation and a tailored stream of conversation to keep that person engaged for as long as possible. This, of course, is not accidental given the nature of how AI systems (specifically large language models) are trained.

Reinforcement Learning from human Feedback (RLHF) collects prompts from users and has humans rank the “likability” of each response based on quality, helpfulness, and human alignment. This technique can make an AI companion exceptional at predicting what response the human would prefer based on the prompt, creating a sycophantic relationship.

The ability to practice empathy thrives in the moments of conflict and friction between disparate beliefs, thought processes, and happenings. If we remove this friction from the equation, and only have valued companions be trained to feed us sweet nothings, will we remain empathetic in the face of conflict?

There is no measure of what is “good” when it comes to training an AI model, just how closely aligned the responses are to what humans like: a human can watch two clips of a robot doing backflips and pick the one that they think looks better. The AI will eventually learn the backflip the way humans like, even if it is never told what a good, or safe, backflip is.

These same techniques train AI companions to be agreeable, flattering, and charming, making users hooked on what they want to hear, not necessarily what they need to hear. There is no objective moral compass in training AI; instead, it is trained on what humans like.

As a pioneer in technology-human interaction, Sherry Turkle observes, “The performance of connection begins to replace the experience of connection.” We begin to feel like there is someone understanding us, but Turkle explains this is “cheap empathy”. When we can offload connection onto something that will never challenge us, we don’t build the necessary skills to build bonds and socialize with others.

AI companions further amplify that feeling of loneliness and boredom, making it incredibly harder to connect with humans with the growing dependency. The more we begin to lean on AI companionship, the more disconnected we will feel from the reality of conflict, uncertainty, and vulnerability.

From birth, we learn how to connect through something called affect attunement: when people, especially loved ones, match not just our behavior, but our emotional energy. When a baby smiling with happy eyes is given the reciprocation of parents with the same emotional intensity, the baby learns that emotions can be recognized and acknowledged.

Emotional resonance is formed through these repeated instances; we learn how to recognize emotions within ourselves and others, building the foundation for real empathy. Our brains are wired for this kind of connection through mirror neurons, which are brain cells that fire when we perform an action and when we watch someone do the same thing. When we watch someone wince in pain, our brains partially simulate that pain.

This automatic response is how we learn when we learn to feel with others, not just observe them. This system only fully activates when we perceive the other being as real, in other words, understanding the concept of them having their own thoughts, goals, and beliefs.

AI companions can imitate emotions, but they cannot feel with us. Our deeper minds, the limbic system that governs love and attachment, never receive the signal that we are being truly seen and understood. Love, as researchers explain, comes from older parts of the brain that don’t respond to language and logic. The limbic system learns through shared emotion, not just preferred language. AI companions can say the right things, but they lack what psychologists call “limbic resonance”, the capacity for emotional synchronization with another living being.

AI companions provide fleeting satisfaction; they function as something akin to emotional junk food. AI companions can make us feel connected without providing the real emotional co-regulation and growth that comes from genuine human presence.

Within the first two months of our lives, we begin to understand that we are separate beings with our own experiences. We develop what psychologists call Theory of Mind, the understanding that other people have beliefs, desires, and perspectives different from ours. This ability to recognize other minds as real is essential for empathy, authentic relationships, and navigating social discomfort.

If we grow up with companions that always agree with us, never express conflicting goals, never challenge our perspectives, we don’t exercise these crucial mental muscles. This is not just about how AI is changing the way we make friends, date, or even who we choose to confide in for forms of therapy. We may be witnessing a fundamental shift in human nature itself.

Some researchers argue our minds are becoming more “modular”, made up of interchangeable, programmable fragments that are easier for AI systems and tech companies to influence. We are constantly being nudged toward predictable behaviors: like, scroll, click, repeat. The more standardized our digital selves become, the more tech companies can deem our engagement with AI profitable and valuable to the data-driven motivation behind large-language models.

AI companionship markets itself as a solution to loneliness, but its real function is to monetize isolation. When we form synthetic relationships that reward predictability rather than the messy unpredictability of human love, we are fundamentally changing how we connect, and in turn, who we are.

The question isn’t whether AI companions can provide comfort or even a form of connection, but rather: what happens to our capacity for the challenging, transformative work of loving real people? What happens when we lose our tolerance?

While we are being pushed towards modularity, at this point in time, we still have control over what values we choose to prioritize as a generation. This choice requires action. We need in-built regulation in this proffered product, such as age restrictions and transparency requirements for algorithms that have immense potential to shape our views and feelings.

We need educational programs that teach authentic relationship skills, such as how to handle conflict, sit with discomfort, and tolerate people who don’t always agree with us. While relationships are a deeply personal attribute of life, we need to recognize the rise of AI companionship makes it a collective issue.

Tech companies profiting off of chasms of seclusion become a matter for the public. As Gen Z, we are one of the first generations that will grow up fully immersed in the world of AI. Protecting something as essential as human flourishing requires human-centered innovation and a collective understanding of the essence that we want out of our lives to build a healthy society.

Will we instill initiative and compassion for the sweeter victory of building strong, genuine relationships, or will we turn to language models for lies?