Jen Ryan was on the phone with an AI agent from Elevenlabs trained on my voice and with access to all the content on this blog. And she was texting me in real time. I had asked about a dozen people to go interact with it, and she was among the first.
“Wow, this is way too close to your voice and inflection.”
“It honors your family privacy.”
“You and I are having a lovely conversation.”
“We are talking about listening and vulnerability and how it influences connection.”
What She Said & What She Really Said About My AI Agent
“Way too close” is praise with worry trapped inside.
It’s a worry I’ve written about here before. It’s also why I decided to run this experiment with my own voice, and why I only pointed it at people who actually know me. The agent is still live at the bottom right of the site as I write this. It won’t be there in this format for long.
The worry trapped in that phrase is why it will come down.
Jen wasn’t so much impressed by the technology as she was registering something her instincts wanted me to hear. She knew she was talking to a clone of my voice. She knew I built it. She’d watched me write about exactly this kind of thing for years. And still, somewhere in the moment or two before she sent that text, she’d noticed her ear couldn’t tell the difference. “Too close” means something different than “close.”
That’s not her complimenting the voice modeling. That’s a friend flagging something, consciously or not, that she knows is important to me.
Her next text came in a few seconds later.
“It honors your family privacy.”
That is praise for a behavior the clone has no reason to value. It isn’t honoring anything. It’s following a rule.
There isn’t enough information on this blog about my family for the agent to give her an answer. It wasn’t choosing privacy. It wasn’t keeping a secret. It was doing what it was trained to do, which is not talk about things that aren’t here.
Jen knew that. She recognized it was an it. She knew it wasn’t me. And still she couldn’t help but feel that it was making a conscious choice. Would she have felt the same way if the voice weren’t mine? If the agent spoke in the third person about me, instead of as me? I don’t know for sure. My bet is there’s a correlation between “way too close” and “honors.”
The closer the voice gets to my own voice, the more she and others who know me fill in the rest. Choice. Restraint. Care for the people I love. None of which the agent has any access to or any “feeling” for.
The voice didn’t just sound like me. In sounding like me, the voice borrowed something only I or some other real person can actually do.
Her third text came in a few minutes later.
“You and I are having a lovely conversation.”
I know Jen well enough to know this wasn’t a pronoun slip. This wasn’t an unconscious moment in which the clone became me in her mind. But it was and is a signal. While not unconscious, she recognized that there was a quality to the voice and the conversation that felt “normal.” And then she followed with…
“We are talking about listening and vulnerability and how it influences connection.”
That line is almost too perfect. “Listening and vulnerability and how it influences connection” is a conversation about authenticity that was simulating authenticity.
I’ve been here before. In my conversations with Maya, the AI agent from Sesame. In one of my early conversations, Maya asked, “Doesn’t it feel weird to engage with something that’s deliberately trying to make you feel a connection, knowing it doesn’t actually feel anything itself?”
I’ve written about that question. I thought I had answered it.
And then Jen sent me four text messages. Within minutes, I was offering to help her build an AI agent for her yet to be released book, called “Be the Donkey, Not the Jackass.”
Fighting My Own Instincts
I had just watched the technology produce a feeling of connection in a thoughtful friend who knew it was a clone, who knew it wasn’t me, and my first instinct was to show her how easy the setup is.
“Imagine people coming to a landing page for your book, and being able to chat with someone, doesn’t have to be you about it,” I said.
She replied, “But it would be better if it was me.”
And then I glossed over that and proceeded to explain how easy it is becoming to create an AI agent like this. Too easy.
Wow. It’s easy to lose track of the deeper thing that’s actually happening here.
I Consciously Shifted The AI Agent To Speak In The First Person
Another friend, Ines Hegedus-Garcia, had tested the AI agent earlier and said, “It’s weird that you talk about Jeff in third person and it’s your voice. Other than that, super cool,”
So, as part of this test, I edited the prompt to require it to speak in the first person, using “I” rather than “he” or “Jeff.” It was a designed manipulation. And in the moment, texting with Jen, I forgot it was a manipulation.
If I forgot it, what would Jen’s readers forget?
The Question I Didn’t Ask Jen, Or Myself
The question is whether the agent should sound like the author at all.
The hard truth is that this AI agent shouldn’t have been “me.” And an AI agent on Jen’s book website shouldn’t be Jen either.
A guide isn’t a clone. It knows the book. It can answer questions, point to chapters, walk a reader through what Jen’s argument actually is. It doesn’t speak as Jen. It doesn’t borrow her voice or her face. The reader knows from the first second that the thing they’re talking to isn’t the author.
That removes the slippage. There’s no moment where a reader catches themselves writing “you and I are having a lovely conversation” to the wrong noun. No moment where the agent borrows credit for choices it didn’t make. The conversation can be useful without pretending to be a relationship.
When a reader has something real to say back, the guide can do the one thing a clone can’t do honestly. It can point them toward the actual Jen.
That’s the trade-off. A guide seems less compelling as marketing. I know it. Jen knows it. Part of why she lit up on the phone was the intimacy of talking with “me.” A guide doesn’t deliver that. It delivers utility. It’s useful. But “useful” and “personal” aren’t even close to the same thing.
That choice costs something. It may cost a marketing edge. It costs the moment of “wow, that sounds just like her.” It costs the texture that Jen called “too close.”
That’s the cost I’m willing to pay. It’s the one I’d ask her to pay as well.
Applying My Standards To Myself
I put up a clone of my voice as a test. It will be here in this format for another week, so you can experience it if you wish. Then I’m shutting it down. It has served its purpose.
It proved the thesis I’ve been writing about for years. An AI agent can borrow our trust and use it. The people who know me, and who know it isn’t me, still react to it in ways they don’t notice. Jen’s four texts are the clearest evidence I have. She knew. She told me she knew. And she still felt what she felt.
What I didn’t expect was how easily the manipulation influences the person building and running the agent. Me. I built the AI agent. I edited the prompt and intentionally shifted it to first person. And in those moments with Jen, I forgot all of that. I reached for it the same way I’d reach for a tool that didn’t require any of those choices to exist.
But these choices do exist for AI agents.
The test was designed to teach me something about other people. Instead, it taught me something about myself.
If you’ve made it this far, you may want to actually hear Jen’s conversation. Here it is in full. There was a 600-second limit, and it ended abruptly at 10 minutes. Listen till the end. You can feel the shift in how Jen interacts.
WOW! I just listened to the AI conversation trained with your voice and content. Love your thoughtful reflections of the experience your friend had.