I’m going to tell you something that most people in Silicon Valley won’t. Something I’ve seen firsthand building and selling two AI companies and investing in over 200 more, including some of the biggest names in the game like Anthropic and OpenAI. Everyone loves to talk about the breakthroughs, the massive funding rounds, the world-changing potential. But nobody talks about the real cost of making it all safe.
And I’m not talking about the cost of compute or the price of a few extra engineers. I’m talking about a cost that’s much harder to quantify, a cost that eats into your budget, your timeline, and your very culture. It’s the cost of human capital, the cost of a safety-first culture, and the opportunity cost of the things you can’t build.
I’ve seen it happen time and time again. A promising AI startup, flush with cash and talent, grinds to a halt because they didn’t factor in the true cost of safety. They thought it was a line item on a spreadsheet, a box to be checked. They were wrong.
The Billion-Dollar Bidding War for a Single Mind
Let me tell you a story. A few years ago, I was advising a startup that was building a new kind of large language model. They had a brilliant team, a great product, and a clear path to market. But they had a problem. They needed a top-tier AI safety researcher, and there were only a handful of people in the world who fit the bill.
We found the perfect candidate, a young woman who had just published a groundbreaking paper on AI alignment. We made her an offer, a very generous offer. She countered with a number that made my jaw drop. It was more than we were paying our CEO.
We were in a bidding war with a tech giant, and we were losing. We ended up losing her. The cost of that one hire would have been in the millions, not to mention the equity. And that’s just one person. A proper AI safety team can have dozens of people, each with a price tag that would make a CFO weep.
This is the human capital cost of AI safety. It’s not just about finding the right people; it’s about being able to afford them. And with the demand for AI talent at an all-time high, that’s a challenge that’s only going to get bigger.
The Culture Clash: Moving Fast vs. Being Safe
But let’s say you manage to hire the dream team. You’ve got the best and brightest in AI safety, and you’re ready to build the future. Now you have a new problem: culture.
A safety-first culture is, by its very nature, a culture of caution. It’s a culture of “what if?” and “have we considered all the angles?” It’s a culture that moves slowly and deliberately. And that’s a good thing, when it comes to safety.
But it’s also a culture that can clash with the “move fast and break things” ethos of Silicon Valley. When your competitors are shipping new features every week, it’s hard to tell your team that you need to spend another month in testing because you’re worried about a one-in-a-million edge case.
I’ve seen this clash play out in my own companies. At RemoteTeam, we were building an AI-powered platform to help companies manage their remote teams. We had a feature that could predict when an employee was at risk of burnout. It was a great feature, but it was also a feature that had the potential to be misused.
We spent months debating the ethics of it. We brought in outside experts, we ran countless simulations, we built in so many safeguards that the feature was almost unusable. In the end, we shipped a watered-down version of it, and it never really took off.
Was it the right decision? I don’t know. But I do know that it cost us time and money. And it created a tension between our engineering team, who wanted to ship a great product, and our safety team, who wanted to ship a safe product.
The Ghosts of Features Unbuilt
And that brings me to the final cost, the one that’s the hardest to measure: the opportunity cost. For every feature you build, there are a dozen you don’t. And when you’re spending a significant portion of your resources on safety, that number gets even bigger.
I think about the things we could have built at MovieLaLa, my second company, if we hadn’t been so focused on safety. We had an idea for a feature that would let you create your own movie trailers using AI. It would have been a game-changer, but the risk of deepfakes was too high.
We had another idea for a feature that would recommend movies based on your emotional state. It would have been a huge technical challenge, but the potential for misuse was too great. We ended up selling to Gfycat before we could ever build it.
These are the ghosts of features unbuilt, the ideas that died in the name of safety. And while I don’t regret the decisions we made, I do wonder what could have been.
The Bottom Line: What Does It Really Cost?
So what’s the bottom line? What does it really cost to do AI safety right? The truth is, there’s no easy answer. It depends on the size of your team, the complexity of your product, and your tolerance for risk.
But I can give you a few numbers to give you a sense of the scale. For a startup with a team of 50, I would estimate that you need to spend at least 20% of your R&D budget on safety. That’s 10 full-time employees, just focused on safety. And that’s a conservative estimate.
For a larger company, that number can be much higher. I know of one tech giant that has a team of over 1,000 people working on AI safety. That’s a billion-dollar-a-year investment.
And that’s just the direct costs. It doesn’t include the indirect costs, like the opportunity cost of the features you don’t build or the cultural cost of a slower, more cautious development process.
A Call for Transparency
I’m not writing this to scare you. I’m writing this because I believe that we, as an industry, need to be more transparent about the real costs of AI safety. We need to stop pretending that it’s a simple problem with an easy solution.
It’s not. It’s a hard problem, a messy problem, a problem that’s going to require a lot of smart people and a lot of money to solve. And we need to be honest about that.
So the next time you read about a new AI breakthrough, I want you to ask yourself a question: what did it cost to make it safe? Because I guarantee you, it was more than you think.
Frequently Asked Questions
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.