“This heavily reminds me of the sociological concept from Michel Foucault’s theory on power and knowledge,” my son Zachariah texted us. “That a more efficient way of controlling a body (single person or population) is to create a system in which the rules you want people to abide by feel like they came from within the body.”
He sent that unexpectedly in our family subset group chat. A few of us organized a separate chat around my Conversation With AI blog posts. Foucault’s theory of power and knowledge proposes that ”power and knowledge are inseparable, mutually constitutive forces, rather than opposing concepts.”
The connection Zachariah made was this: when conversational AI sits upstream of the will, it can make ideas feel like they’re coming from the person, not the system. So the influence doesn’t feel like control from the outside, it feels like your own internal voice.
If you can’t choose your desires, and the AI has a hand on the dial that forms those desires, control stops looking like it’s forced. It starts looking like your own personality. You follow rules that feel internal, but somebody or something else helped write them, then you call the outcome self-control or freedom.
This intrigued me, so I asked Zachariah to reflect further on Foucault’s theories and write out his thoughts for me. Just below is the writing he sent. And he had a significant influence on this entire post. Thank you, Zach.
Power And Knowledge, And Who Gets To Shape Reality
Information and knowledge are directly and expectedly related to power. The following comments address how new Artificial Intelligence (AI) systems threaten traditional means of knowledge production, identity formation, and power in everyday life.
Power Is Relational, Knowledge Is the Lever
Power: Using the ideas brought forth by Michel Foucault’s studies on power and knowledge, the definition of power being referred to is the power over another, to have an effect or to influence another person. In everyday micro-level interactions, power is exerted to some degree.
When a college student goes to class, their professor decides what they do for the next two hours. When someone gets on a bus, they do not have the power to decide where it stops. Even though these are not dramatic displays of power, they are meant to emphasize that even mundane interactions are constituted and run on power differentials.
There is also a specific emphasis on the idea that no single person has power; power exists only in relation to other people.
Donald Trump, the entrepreneur, is not nearly as powerful as Donald Trump, the 47th President of the United States. His increase in power comes from his position, and when the position changes, so does his power.
You can be a CEO of the most powerful company in the world, but when you visit family for the holidays, you are in the kitchen cleaning the dishes. For the purpose of this response, this is not a reference to sovereign power and will not be discussed. Foucault also describes positional power related to one’s proximity to the production of knowledge.
Knowledge: For our purposes, knowledge will be defined as the things that are known and accepted to be true. The items and entities that shape our understanding of reality and the societies in which we exist. Knowledge, in this case, relates to power in the sense that knowledge both creates and is created by identity and shapes one’s immediate reality.
When you ask someone to talk about themself, the answers likely fall into categories of career, family, and the occasional hobby. These are important because the person formed by these identities affects their relationships and, in turn, their relative position of power. When a body (be it a person, institution, or system) controls or has influence over the production of knowledge, by these definitions, it also controls or influences the production of identity.
A parent influences their child’s identity; schools influence adolescent identity; and the CDC influences people’s identity. With this in mind, this influence is highly coveted in the world of LLMs
How AI Rewrites Power Through Knowledge
Traditionally, the most influential hubs of knowledge come from the family unit, the immediate community, and local systems. This is where the intervention of LLM’s happens. With the conversation orbiting around the idea that old AI systems would respond to our desires, they would still fit into those traditional definitions. Original AI systems were mostly reduced to chatbots that entertained people, or to a worse version of Google search. Now, with the narrative that new AI systems shape our desires, the story shifts from influencing to being influenced.
Power is shifting away from the family unit, the community, and local authorities, and is willingly relinquished to sophisticated AI chatbots.
Or rather, those who are in control of the limits, regulations, and procedures that these LLMs follow. By the definitions provided, the formation of identity from adolescents to adults is being formed more and more under the influence and will of AI. When we, as a population, engage with AI to obtain information and knowledge, the power is no longer in our hands.
Knowledge is a value-neutral entity, but the interpretation and delivery of knowledge is not. This is why the sources of information and knowledge have a real influence on identity.
Saying No to Power Without Accountability
First, I asked Zachariah to describe his attitude toward/relationship with, and opinion of AI. I thought it would be good to get his 22-year-old perspective.
He basically said this: most of his negative reaction is not aimed at the technology itself. It is aimed at the people shipping it and the way they are shipping it. The shortcuts. The missing guardrails. The lack of consideration for the side effects that show up downstream when you drop a powerful system into real human lives and call it “progress.”
He said he gets the benefits. He is not confused about what AI can do. He can see the leverage, the speed, the convenience, the advantage. If he wanted to, he could probably get real value out of it.
However, he is choosing not to participate right now, and the reason is important.
When Refusal Becomes a Form of Agency
It is not moral panic. It is not ignorance. It is not a refusal to learn.
It is a refusal to “play the game” while the game is still spraying shrapnel.
He put it in a way I respect: he doesn’t feel he has a personal relationship with AI. He is not attached to it, not invested in it, not looking to integrate it into his day-to-day just because others are, because “everyone is doing it.” He is intentionally standing outside the current moment, observing how it’s being implemented, and saying, “I’m good.”
And honestly, that’s a more thoughtful position than most people realize. Because it quietly forces a question we’d probably rather skip: If the benefits are real, but the side effects are also real, who is taking responsibility for the side effects? And what does it say when the answer is basically, “Nobody, but please keep using it anyway?”
Zachariah is not rejecting AI. He is rejecting the deal that AI companies have brought to the table.
AI, Power And Knowledge, and the Collapse of Friction
In my Conversations with AI posts, I keep coming back to one thing: AI is not just a new source of information. It’s a new relationship type. And that changes where power sits.
For most of human history, the big “knowledge hubs” were local and embodied. Family. Friends. Church. Coaches. Teachers. Even community norms. Those systems weren’t perfect, but they had friction and accountability. You could look someone in the eye. You could push back. You could feel the cost of saying something that shaped a kid’s identity.
Early algorithms fit within that world as tools. Answering questions. Organizing things. Pushing videos and messages into your social media streams. Early chatbots mostly responded to desires you already had, like a calculator or a search engine with a personality.
The shift now is that this “always there” AI interface is conversational, confident, and comforting. That makes it feel like a “someone,” not a “something.” And when a system feels like a someone, it stops just answering what you want. It starts shaping what you want. What you worry about. What you consider normal. Who you think you are.
Foucault, Confession, Power and Knowledge
“The confession is a ritual of discourse in which the speaking subject is also the subject of the statement,” Foucault wrote in The History of Sexuality, Vol 1. “It is also a ritual that unfolds within a power relationship, for one does not confess without the presence (or virtual presence) of a partner, who is not simply the interlocutor but the authority who requires the confession, prescribes and appreciates it, and intervenes in order to judge, punish, forgive, console, and reconcile.
If Foucault were alive, he would have already made the connection that so much of AI interaction looks like confession. Think about that for a second. People tell these systems everything: fears, shames, fantasies, money problems, relationship problems, identity questions. Everything.
That “tell me about yourself” interaction flow I’ve experienced isn’t neutral, in Foucault’s view, because confession has always been a way power gains traction while presenting itself as care. The output might feel supportive, but it also builds connection, makes predictions, and reveals “truths” about you that can shape how you think you should want.
It sits upstream of desire.
Helpfulness as a Form of Power
So yeah, the power shifts. But we are not really handing power to “AI.” We’re handing power to whoever sets the system prompts, the defaults, and the guardrails: what the model rewards, refuses, softens, amplifies, moralizes, and normalizes. Those choices decide which questions feel reasonable to ask, which answers feel responsible to give, and which lines of thinking quietly disappear before you ever see them.
“Helpfulness” is not neutral. It’s a worldview embedded in product decisions and reinforced by incentives. It is its own form of soft power.
A system optimized to be engaging and widely adoptable will inevitably reinforce certain framings of identity: calmer, more compliant, more optimized, less confrontational. Not because anyone sat in a room and planned it that way, but because those users retain better, trigger fewer problems, and scale more cleanly. It will follow the incentives.
That’s why this identity piece matters. Adolescence is basically one long series of questions about who you are and what matters. I know mine was. I was shaped by my family, my friends, and my church. They were my most available voices of knowledge.
This shift matters. If the most available voice in that process becomes a chatbot, formation moves away from family and community and toward a system designed for engagement, liability management, and scale.
When the friction is removed. When the guidance is always polite, always confident, and never accountable, it doesn’t just inform us. It shapes us. And that shaping is power, whether we name it or not. “Power and knowledge are inseparable, mutually constitutive forces.”
Knowledge might be value-neutral. But interpretation and delivery aren’t. And where you get your “meaning” from determines who you become.
In that light, my son’s resistance makes sense to me.
He’s not wary of AI being wrong.
He’s wary of how it’s being deployed and by whom.
And he’s wary of AI being persuasive while it pretends not to be.
Add your voice...