I’ve been kicking around Silicon Valley long enough to see hype cycles come and go. I’ve built and sold companies, and now I spend most of my time investing in the next generation of builders, including some of the names you see in the headlines every day like Anthropic, Scale AI, and OpenAI. And let me tell you, the most interesting stuff, the real alpha, is never what everyone is talking about.
Right now, the world is obsessed with AI models. GPT-4, Claude 3, Sora—they’re the front-page news. But that’s like admiring a skyscraper’s spire without ever thinking about the foundation. The real action, the brutal, high-stakes game that will determine the next decade of AI, is happening in the silicon. It’s a battle fought with fabs, interconnects, and esoteric architectures. This is the hardware that actually breathes life into the algorithms, and the truths of this world are far from obvious.
I’ve been in these trenches. I’ve seen brilliant startups get kneecapped by the GPU shortage, and I’ve backed teams who were crazy enough to make their own chips. I’m writing this because the common wisdom about AI hardware is dangerously simplistic. It’s time for an unfiltered look at what’s really going on.
The GPU is Not the End-All, Be-All
Let’s get this out of the way: NVIDIA. They’re on top of the world, and they’ve earned it. The H100 is a beast, and their CUDA software ecosystem is a fortress. For years, the default answer for any AI company has been “just buy more NVIDIA GPUs.” But that’s not a strategy; it’s a reflex. And it’s becoming a lazy, expensive, and dangerous one.
I was advising a promising startup a couple of years back. They had a brilliant idea for a new video analysis platform. They’d raised a solid seed round, but their burn rate was terrifying. I dug into their financials, and the culprit was obvious: their cloud bill was a monument to NVIDIA. They were spending nearly $150,000 a month on the latest and greatest GPUs, convinced it was the only way. They were so fixated on the H100s they couldn’t get, they were overpaying for any GPU they could find.
I pulled the founders into a room and put it bluntly: “You’re not a hardware company, you’re a software company. Why are you lighting money on fire for hardware you don’t need?” We spent a weekend whiteboarding their entire stack. We found that 80% of their workload was inference, not training. And for their specific model, older generation A6000s, or even AMD’s MI250s, could deliver 90% of the performance for less than half the cost. We also looked at FPGAs for some of their highly specialized pre-processing algorithms. It was a complete shift in mindset.
It took them a quarter to re-architect their platform, but the result was a 60% reduction in their cloud bill. That wasn’t just savings; it was an extra 18 months of runway. It saved their company. This is the first counterintuitive truth: The best chip is not the most powerful chip. It’s the right chip for the job.
The Rise of Custom Silicon
This leads to the next big truth: the explosion of custom silicon. For most of my career, telling a VC you were building your own chip was a great way to get laughed out of the room. The NRE (non-recurring engineering) costs were astronomical, the design process was a multi-year slog, and you were competing with giants like Intel. That’s not the world we live in anymore.
Look at the landscape. Google has their TPUs, Amazon has Trainium and Inferentia, and Tesla is all-in on their Dojo chip. These aren’t vanity projects. They are strategic necessities. These companies realized that to win at their scale, they couldn’t depend on an external roadmap. They needed to control their own destiny. The tools from Cadence and Synopsys are now more accessible, and TSMC’s multi-project wafer runs allow startups to get their first silicon in a shared, more affordable way. The rise of open-source architectures like RISC-V is also a huge catalyst.
I’ve personally invested in two custom silicon startups. One is building a chip for large-scale physics simulations, a workload where traditional GPUs are shockingly inefficient. The other is focused on a novel architecture for running transformers. I didn’t back them because I think they’ll kill NVIDIA. I backed them because they’re not even playing the same game. They are building for a specific, high-value niche where they can be 10x better, not just 10% faster.
This is a trend that’s only going to accelerate in the coming years. And it’s going to have a profound impact on the AI landscape. It’s going to level the playing field, allowing smaller companies to compete with the big guys. And it’s going to lead to a new wave of innovation, as companies are freed from the constraints of off-the-shelf hardware.
The Edge is Where the Action Is
The cloud gets all the attention, but the edge is where AI gets real. The edge isn’t just a smaller cloud; it’s a completely different universe. In the data center, you worry about TCO and performance-per-watt. On the edge, you worry about a 5-watt power budget and whether your chip will overheat in a sealed enclosure in the middle of a desert.
This is where the real creativity in chip design is happening. We’re talking about neuromorphic chips that mimic the brain’s architecture, in-memory computing that eliminates the bottleneck between processing and memory, and radical new approaches to quantization that can shrink a massive model onto a tiny piece of silicon. The team at one of my portfolio companies is working on a chip for smart cameras that can perform complex object tracking using less power than a Christmas light. That’s not something you can do with an off-the-shelf GPU.
I’ve seen some incredibly innovative solutions in this space. Companies that are using new materials, new architectures, and new manufacturing techniques to build chips that are smaller, faster, and more power-efficient than anything that’s come before. This is where the real cutting-edge research is happening, and it’s a space that I’m watching very closely.
The Data Center is Not Dead
Now, don’t get me wrong. The data center isn’t going anywhere. Training the next generation of foundation models will require more flops than ever. But the data center of tomorrow will look nothing like the ones we have today.
We’re entering an era of co-design and specialization. The most advanced AI teams are no longer just writing software; they’re designing the hardware and software together. They’re thinking about how the physical layout of the chip affects the performance of their neural network, and how the network architecture can be optimized for the interconnects. We’re seeing the rise of optical interconnects to move massive amounts of data between chips, and liquid cooling is becoming standard, not an exotic luxury. The future is heterogeneous, with racks containing a mix of GPUs, TPUs, and other custom accelerators, all working in concert.
The Real Bottleneck: Talent
There’s one last truth that nobody talks about. The biggest constraint in AI hardware isn’t silicon, or manufacturing capacity, or even money. It’s talent. There are maybe a few hundred people in the world who can lead the design of a truly cutting-edge chip. And they are in a bare-knuckle brawl for talent. I saw one of my portfolio companies spend six months and a small fortune trying to hire a lead verification engineer. It’s a brutal, zero-sum game. The war for AI will be won not just with better chips, but with the brilliant, obsessive, and slightly crazy people who design them.
The Unfiltered Truth
So what does this all mean? It means the age of lazy hardware decisions is over. You can’t just throw money at NVIDIA and hope for the best. You need to think like a system architect. You need to understand your workload, your bottlenecks, and your unit economics.
Do you need the absolute fastest training time, or is inference cost your biggest driver? Are you building for the cloud, or for a device that runs on a battery? Are you prepared to invest in the software and talent to support a more diverse hardware stack? These are the hard questions that founders need to be asking.
The future of AI won’t be built on monolithic hardware. It will be built by teams who are smart, scrappy, and willing to challenge the status quo. It will be built on a diverse, heterogeneous, and custom-built foundation. The real innovation is just getting started. Now go build it.
Frequently Asked Questions
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.