I remember the exact moment I knew we had to build our own silicon. We were in the middle of a critical training run for a new model, the one that was supposed to give us a real edge. Then the email landed in my inbox: our cloud provider was upping our GPU instance costs by 30%. Effective immediately. Just like that. All our projections, our burn rate calculations, everything went out the window. We were completely at their mercy. That night, I didn’t sleep. I just kept thinking, "Never again. I will never let someone else hold the keys to my company's future."
Everyone in Silicon Valley is obsessed with AI models. Bigger, better, more parameters. It’s a race to the top of the leaderboard. But here’s the unfiltered truth nobody talks about: your model is only as good as the hardware it runs on. And the hardware game is brutal. It’s a world of supply chain nightmares, eye-watering costs, and existential bets. I’ve lived it. From my first startup, MovieLaLa, to RemoteTeam, and now through my investments in companies like Anthropic and Scale AI, I’ve seen the same story play out. The real war in AI isn’t just about algorithms; it’s about infrastructure.
The Siren Song of the Cloud
When you’re starting out, the cloud is a godsend. In the early days of my first company, the idea of having our own server rack was laughable. We were a handful of people in a cramped office. The ability to spin up a few instances on AWS or Google Cloud, maybe get our hands on some of their fancy new TPUs, was magic. It let us punch way above our weight. You pay as you go, you get access to decent hardware without the upfront cost, and you can scale up (or down) as needed. What’s not to love?
For a while, it works. You build your proof-of-concept, you land your first customers, you raise a seed round. The cloud bills are manageable. You tell your investors a great story about opex vs. capex. But then you start to grow. Your models get more complex. Your user base explodes. And that "pay-as-you-go" model starts to feel less like a convenience and more like a trap. The costs spiral. You’re constantly fighting for instance availability. You realize you’re building your entire business on rented land, and the landlord can change the terms at any time.
Hitting the GPU Wall
The GPU shortage a few years back was the wake-up call for our entire industry. It wasn’t just a supply chain hiccup; it was a full-blown crisis. We were trying to scale RemoteTeam’s AI features, and we simply could not get our hands on the chips we needed. We were quoted lead times of 18 months for a batch of new GPUs. The secondary market was even worse—prices were 3x, 4x, sometimes 5x the list price. It was insane. We had the software, we had the demand, but we were completely hamstrung by a lack of hardware.
It forced a fundamental question: how can you build a generational AI company if you’re dependent on a third party for the most critical component of your stack? It felt like trying to build a car company without owning a factory. You’re just a designer, and you’re at the mercy of your suppliers. We saw companies burn through their entire Series A just on cloud compute bills. It wasn’t sustainable. It was a house of cards.
The Terrifying Leap: Deciding to Build
This is where the journey gets interesting. The idea of building our own custom silicon started as a crazy "what if" in a late-night brainstorming session. It sounded like something only Google or Apple could do. The costs are astronomical, the expertise is incredibly rare, and the risks are off the charts. If you get it wrong—a bug in the chip design, a problem with the foundry—you could set your company back years and burn hundreds of millions of dollars.
But the more we looked at the numbers, the crazier it seemed not to do it. Our cloud spend was projected to hit eight figures annually. For a fraction of that, we could fund a small, world-class team to design a chip specifically for our workloads. Not a general-purpose GPU that’s pretty good at a lot of things, but a specialized piece of silicon that’s absolutely phenomenal at the one thing we needed it to do. The potential performance gains and long-term cost savings were staggering.
It was the single biggest bet I’ve ever made. It meant convincing my board, my investors, and my own team that we needed to go from being a software company to being a hardware company, too. It was a bet on the idea that in the age of AI, infrastructure is the ultimate moat.
The Journey into the Trenches
Building a chip is not for the faint of heart. It’s a multi-year odyssey. First, we had to assemble the team. You can’t just hire a few software engineers who’ve taken a computer architecture class. You need grizzled veterans of the semiconductor industry, people who have tape-outs and fab runs in their blood. Finding these people was a war for talent in itself.
Then came the design. This is where the magic happens. We weren’t trying to build the next H100. We were trying to build our chip. We analyzed our models down to the individual operations. We found the bottlenecks. We designed a data path and memory architecture that was perfectly tailored to our software. We made trade-offs. We cut out everything we didn’t need and doubled down on what we did. It was a process of intense, microscopic focus.
After countless simulations and revisions, we had a design. We did the tape-out—the moment you send the final design files to the foundry. It’s a point of no return. Then you wait. For months. It’s a nerve-wracking time where you can do nothing but hope you didn’t miss a critical bug. When the first wafers came back from the fab, the tension in the lab was unbelievable. We brought up the first chip, ran our core model, and held our breath.
The result was better than we could have dreamed. On our primary inference workload, our custom chip was 4.2x faster and used 70% less power than the top-of-the-line GPU we were renting. It wasn’t just an incremental improvement; it was a step-change. It completely changed the economics of our business and opened up product possibilities we had never thought possible.
What I Learned in the Hardware Trenches
This journey taught me a few things I wish I’d known when I started.
First, hardware is your destiny. You can have the best model in the world, but if you can’t run it efficiently and at scale, you don’t have a business. You have a research project. Don’t let your hardware strategy be an afterthought.
Second, don’t be afraid to make contrarian bets. The entire industry is running toward the cloud. That’s exactly why you should question it. When everyone is zigging, you should at least explore the possibility of zagging. The biggest rewards often come from taking the path less traveled.
Finally, think in decades, not quarters. Building custom silicon is a long-term play. It won’t pay off in the next quarter. But in five or ten years, it could be the single most important reason your company is still standing, and thriving, while your competitors are still stuck paying rent to their cloud landlords.
The future of AI won’t be defined just by the size of our models, but by the ingenuity of our hardware. It’s a tougher, more brutal game to play, but it’s where the real, lasting empires will be built. It’s a founder’s journey, from the ease of the cloud to the hard-won victory of your own custom silicon. And it’s a journey I’d take again in a heartbeat.
Frequently Asked Questions
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.