I almost lost $50,000 to a phone call. It sounded exactly like my co-founder, panicked and asking for an urgent wire transfer to a new vendor. The only reason I didn’t send the money was because I had a rule: any wire over $10,000 requires a video call confirmation. It turned out to be a deepfake, a sophisticated AI-generated voice clone. That was the moment AI voice stopped being a cool toy and became something I had to master, both as a founder and an investor.
I’ve been in the tech game for a while now. I’ve built and sold two companies, RemoteTeam and MovieLaLa, and I’ve been fortunate enough to write angel checks for over 200 startups, including some of the names you see in the headlines every day like OpenAI, Anthropic, and Scale AI. I’ve seen a lot of tech waves come and go. Some are ripples, some are tsunamis. AI voice is a tsunami.
But it’s not a simple one. For every story of a breakthrough in conversational AI, there’s a story like mine. For every startup cutting costs with an AI-powered phone system, there’s another one alienating customers with a robotic, frustrating experience. I’ve seen both sides up close, and I’m here to share what I’ve learned from the trenches.
The Early Days: More Robot than Human
My first real attempt at using AI voice was back at RemoteTeam. We were growing fast, and our small HR team was swamped with repetitive questions from employees about payroll, benefits, and company policies. The idea was simple: build an internal AI assistant that could answer these common questions over the phone or via a chat interface. We thought we could free up our HR team to focus on more strategic initiatives.
We tried one of the early-generation platforms. The demo was impressive. The voice sounded smooth, the responses were quick. We spent weeks feeding it our entire knowledge base, meticulously crafting question-and-answer pairs. The result? A complete disaster.
The voice, which sounded so good in the demo, was jarring and unnatural when it tried to pronounce the names of our health insurance plans. It would get stuck in loops, endlessly repeating “I’m sorry, I don’t understand the question.” Employees hated it. They found it faster to just wait for an email response from a human. We scrapped the project after three months. It was a humbling experience, and a costly one. We wasted about $25,000 in subscription fees and countless hours of engineering time.
That failure taught me a critical lesson: the uncanny valley is very real for voice. A slightly-off voice is worse than a completely robotic one. It creates a sense of unease and frustration that kills the user experience. For a while, I wrote off AI voice as a gimmick, something that was perpetually “five years away” from being truly useful.
The Breakthrough: When AI Found Its Voice
It took a few years, but the technology started to catch up. My “aha” moment came from an unexpected place: a portfolio company in the podcasting space. They were struggling to produce high-quality audio content at scale. Hiring voice actors for every article they wanted to turn into an audio version was expensive and slow. They were on the verge of abandoning the idea altogether.
On a whim, I connected them with a startup I had just invested in, one that was working on a new generation of voice cloning technology. This wasn’t the robotic voice of the past. This was different. You could feed it just a few minutes of someone’s speech, and it could generate new audio that was almost indistinguishable from the original. The intonation, the pacing, the subtle emotional cues—it was all there.
The podcasting company ran a pilot. They took their top 10 most popular articles and created AI-narrated versions using a clone of their CEO’s voice. The results were staggering. Engagement on their site went up by 40%. The average time a user spent on their platform doubled. They could now convert a 2,000-word article into a professional-sounding podcast episode in under an hour, for a fraction of the cost of hiring a voice actor.
This was the turning point for me. I saw firsthand that when applied to the right problem, AI voice wasn’t just a cost-saver; it was a growth engine. It could create entirely new ways for businesses to engage with their audience. I started looking at all my portfolio companies and asking, “How can we use this?”
The Founder's Playbook for AI Voice
I’ve seen founders make the same mistakes I did, and I’ve seen others knock it out of the park. Here’s my playbook for getting it right:
- Start with the 'Why'. Don't use AI voice just because it's trendy. Have a clear business problem you're trying to solve. Is it about cutting costs? Improving customer experience? Creating new content? Your goal will determine the type of technology you need.
- Nail the Use Case. AI voice is not a one-size-fits-all solution. A voice for an internal HR bot is very different from a voice for a branded podcast. Be specific about the context. Is it for short answers or long-form narration? Is the tone meant to be informative, entertaining, or empathetic?
- Prioritize Voice Quality. Don't cheap out on the voice. A bad voice will kill your project before it even gets off the ground. Listen to demos. Test it with your own content. If it sounds even slightly off, walk away. The technology is getting better fast, but there's still a wide range of quality out there.
- Own Your Voice. Whenever possible, use a custom voice clone. It could be your CEO, a brand ambassador, or a professionally hired voice actor. A unique, consistent voice is a powerful branding tool. It builds trust and familiarity. Generic, off-the-shelf voices make you sound like everyone else.
- Think in Conversations, Not Commands. The best conversational AI doesn't just answer questions; it holds a conversation. It can handle interruptions, ask clarifying questions, and remember the context of the discussion. This is where companies like Anthropic are pushing the boundaries. Look for systems that have a sophisticated natural language understanding (NLU) engine under the hood.
The Dark Side: My Stance on Ethics
We can't talk about AI voice without talking about the dark side. My deepfake phone call was a wake-up call. The same technology that can create a beautiful podcast can also be used to scam, to spread misinformation, and to harass. As founders and investors, we have a responsibility to build safeguards into the technology we create.
I have a simple rule for my investments in this space: if you don't have a clear and robust ethics policy, I'm not writing a check. This includes things like watermarking AI-generated audio, requiring explicit consent for voice cloning, and having a zero-tolerance policy for malicious use. We are at a critical juncture. The decisions we make now will shape the future of this technology. We can't afford to get it wrong.
The Future is Spoken
Despite the risks, I'm incredibly bullish on the future of AI voice. I believe we are moving away from a world of screens and keyboards and into a world where our primary interface with technology will be our voice. Think about it. It's the most natural way for us to communicate.
I predict that within the next three years, AI-powered audio will be bigger than video. Every article, every newsletter, every report will have an audio version. Podcasting will become even more democratized, with individuals able to create high-quality content without expensive equipment. In-car assistants will become true co-pilots, capable of complex conversations and tasks.
For startups, this is a massive opportunity. The next generation of billion-dollar companies will be built on voice. They will be the ones who crack the code of creating natural, engaging, and trustworthy voice experiences. It’s not about replacing humans, but about augmenting them. It’s about using AI to give everyone a voice.
I’m putting my money where my mouth is. I’m actively investing in startups that are pushing the boundaries of what’s possible with AI voice. It’s a wild ride, but I’m convinced we’re on the cusp of a major shift in how we interact with technology. The robots are finally learning to talk, and I’m listening closely.
Deeper Dive: The Technical Hurdles We Faced
Let's get into the weeds a bit on that first failed project at RemoteTeam. The problem wasn't just that the voice was robotic. The real killer was the latency and the lack of context. When an employee asked a multi-part question, the system would forget the first part of the question by the time it was answering the second. It was like having a conversation with someone with severe short-term memory loss.
We tried to solve this by engineering complex state management systems, essentially trying to bolt on a memory for the AI. It became a tangled mess of code that was impossible to maintain. Every time we updated our benefits information, we had to spend days re-training and re-scripting the conversation flows. The NLU models of that era were just too primitive. They were good at recognizing simple keywords and intents, but they couldn't handle the nuances of human conversation. They couldn't understand context, sarcasm, or follow-up questions. We learned the hard way that a good voice is only half the battle. The brain behind the voice is what truly matters.
Another huge technical challenge was integration. Getting the AI voice system to talk to our existing HR software was a nightmare. The APIs were clunky and poorly documented. We ended up building a custom middleware layer just to pass information back and forth. This added another point of failure to an already fragile system. My advice to founders now is to look for platforms that have pre-built integrations with the tools you already use. Don't underestimate the cost and complexity of custom integration work.
My Investment Thesis for AI Voice
When I evaluate a startup in the AI voice space for an investment, I look for a few key things beyond just a cool demo. I want to see a deep understanding of the underlying technology and a clear vision for how to solve a real-world problem. Here are some of the questions I ask:
- What is your data advantage? The best AI models are built on massive, high-quality datasets. I want to know where the startup is sourcing its training data and how they are ensuring its quality and diversity. A model trained only on a specific accent or demographic will fail in the real world.
- How do you handle the long tail? It's easy to build a system that can answer the 100 most common questions. But what about the thousands of less common, more complex questions? I look for startups that have a strategy for handling this long tail of user queries. This might involve a human-in-the-loop system, where the AI can escalate complex questions to a human agent and then learn from that interaction.
- What is your moat? The AI voice space is getting crowded. I want to know what makes the startup defensible. Is it proprietary technology? A unique dataset? A strong brand? A deep partnership with a major player in the industry? Simply wrapping an existing API is not a sustainable business model.
- How are you thinking about ethics? As I mentioned before, this is a non-negotiable for me. I want to see a proactive and thoughtful approach to the ethical implications of the technology. This isn't just about having a policy; it's about building ethics into the product from the ground up.
The Human Element
It's easy to get caught up in the technology, but we must never forget the human element. The goal of AI voice is not to replace human connection, but to enhance it. A well-designed AI voice system can free up humans to focus on what they do best: building relationships, solving complex problems, and providing empathy.
I recently saw a powerful example of this in a healthcare startup I advise. They are using a conversational AI to help elderly patients manage their medications. The AI calls them every day, reminds them to take their pills, and asks them about any side effects. The voice is warm, patient, and empathetic. It can even detect signs of distress in the patient's voice and escalate the call to a human nurse if needed. This isn't replacing the nurse; it's giving the nurse superpowers. It's allowing her to monitor hundreds of patients at once and focus her attention on those who need it most.
That's the future I'm excited about. A future where technology doesn't alienate us, but brings us closer together. A future where AI voice is a tool for empowerment, for connection, and for good. It's a future I'm proud to be building, one investment at a time.
Frequently Asked Questions
Can these results be replicated?
The specific numbers will vary, but the underlying patterns and principles are transferable. The key is understanding the context behind the results, not just copying the tactics. Every company has unique constraints that shape what works.
What would you do differently looking back?
I'd move faster on the things that were working and cut the things that weren't sooner. Most founders, myself included, hold onto failing strategies too long because of sunk cost. Speed of learning is everything.
How long did it take to see results?
Most meaningful business results take 3-6 months to materialize. Anyone promising overnight success is selling something. The companies in my portfolio that grew fastest were the ones that stayed patient and consistent.