I’ve been in the Silicon Valley game for a long time. I’ve built four companies, sold two of them, and invested in over 200 startups, including some of the biggest names in AI like Anthropic and OpenAI. I’ve seen trends come and go, and I’ve learned a lot of hard lessons along the way. And I’m here to tell you that one of the biggest lessons I’ve learned is that the cloud is not always the answer, especially when it comes to AI.
For years, we were all told that the cloud was the future. It was scalable, it was flexible, and it was cost-effective. And for a while, it was. But as AI has become more and more demanding, the cloud has started to show its cracks. The costs have skyrocketed, and the performance has become a bottleneck. I saw this firsthand with my own startups. We were spending a fortune on cloud services, and we still weren’t getting the performance we needed. So I decided to do something radical. I ditched the cloud and went back to bare metal.
The Brutal Reality of AI Hardware
Everyone is talking about AI models, but nobody is talking about the brutal reality of the hardware that runs them. The truth is, the hardware is just as important as the software, if not more so. And right now, we’re in the middle of a massive GPU shortage. The demand for GPUs is far outpacing the supply, and the prices have gone through the roof. This is making it incredibly difficult for startups to compete. They can’t get their hands on the hardware they need, and they can’t afford the cloud services that have the hardware.
I saw this with one of my portfolio companies. They were a promising young AI startup with a brilliant team and a groundbreaking product. But they couldn’t get enough GPUs to train their models. They were stuck in a holding pattern, unable to move forward. It was a frustrating situation, and it’s one that a lot of startups are facing right now.
The Hypervisor Tax
One of the biggest problems with the cloud is what I call the “hypervisor tax.” The hypervisor is the software that creates and runs virtual machines. It’s what allows cloud providers to pack multiple customers onto a single server. But it also adds a layer of overhead that can significantly impact performance. For most applications, this isn’t a big deal. But for AI, where every ounce of performance matters, it can be a killer.
When you’re running on bare metal, you’re not paying the hypervisor tax. You’re getting direct access to the hardware, and you’re getting every last drop of performance out of it. This can make a huge difference in training times and inference speeds. In our case, we saw a 3x improvement in performance when we moved from the cloud to bare metal.
The Contrarian Approach
I know what you’re thinking. “Bare metal is a pain to manage.” And you’re right, it can be. But it’s not as hard as you think. And the benefits can be well worth the effort. There are a number of companies that are now offering bare metal cloud services, which give you the best of both worlds. You get the performance of bare metal with the convenience of the cloud.
I’m not saying that everyone should ditch the cloud and go back to bare metal. The cloud still has its place. But I am saying that you should think twice before you blindly follow the herd. Don’t be afraid to take a contrarian approach. Sometimes, the old ways are the best ways.
The Future is Custom Silicon
Looking ahead, I believe that the future of AI hardware is custom silicon. As AI models become more and more specialized, we’re going to need specialized hardware to run them. This is where TPUs (Tensor Processing Units) come in. TPUs are custom-built chips that are designed specifically for AI workloads. They’re much more efficient than GPUs, and they can provide a significant performance boost.
I’m so bullish on custom silicon that I’m actually building my own. I’ve assembled a team of some of the best chip designers in the world, and we’re working on a new type of AI accelerator that I believe will change the game. It’s a risky bet, but I’m confident that it will pay off.
The Takeaway
My journey from the cloud to bare metal has been a long and winding one. But it’s taught me a valuable lesson. Don’t be afraid to challenge the status quo. Don’t be afraid to go against the grain. And don’t be afraid to build your own future. The world of AI is still in its infancy, and there’s a lot of room for innovation. The ones who will succeed are the ones who are willing to take risks and think differently.
So, if you’re a startup founder who is struggling with the cost and performance of the cloud, I urge you to consider bare metal. It’s not for everyone, but it just might be the answer you’re looking for. And who knows, you might even save 62% on your bill.
Frequently Asked Questions
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.