I remember the exact moment I realized the AI revolution wasn't just about software. It was 2021, and I was trying to secure a large batch of NVIDIA A100s for a portfolio company. The waitlist was a year long. A full year. For a company burning through cash and racing against competitors, a year might as well be a decade. We ended up paying a 3x premium on the secondary market just to keep the lights on. That experience taught me a lesson that has shaped my investment thesis ever since: AI models get all the hype, but the real challenge, the real moat, is the hardware behind them.
Since then, I’ve had a front-row seat to the AI hardware race, both as an investor in companies like Anthropic and OpenAI and as an advisor to startups building the next generation of infrastructure. I’ve seen brilliant teams with innovative models fail because they couldn’t get enough compute. I’ve also seen scrappy startups outmaneuver incumbents by being smarter about their hardware stack. The GPU shortage was just the beginning. The next five years will see a radical transformation in the world of hardware. Here are the six trends I’m watching closely, the ones I believe will define the winners and losers in the race to build true artificial intelligence.
1. The Great GPU Scramble & The Rise of Custom Silicon
Let's be honest, for the last few years, NVIDIA has been the only game in town. Their CUDA software ecosystem is a powerful moat, and their GPUs are the gold standard for training large models. But their dominance has created a massive bottleneck. The scramble for GPUs has become a zero-sum game, and it's forcing a change. We're seeing a Cambrian explosion of custom silicon, and I believe this is the most important trend in AI right now.
Companies like Google with their TPUs, Amazon with Trainium and Inferentia, and Microsoft with Maia are leading the charge. They realized that at their scale, they can’t be dependent on a single supplier. They are designing chips specifically for their own workloads, and the performance gains are staggering. I’ve seen internal data from one of these hyperscalers showing a 40% reduction in training costs for their largest models compared to using off-the-shelf GPUs. That’s a massive competitive advantage.
But it’s not just the giants. I’m seeing a wave of startups building custom ASICs for specific AI tasks. One of my recent investments is a company building a chip for protein folding simulations. By co-designing the hardware and software, they’re achieving a 10x speedup over the fastest GPUs. This is the future. The days of one-size-fits-all hardware are numbered. The winners will be those who can build or access custom silicon tailored to their specific needs.
2. Neuromorphic Computing: The Brain as a Blueprint
For all their power, today’s AI models are incredibly inefficient. A model like GPT-4 consumes megawatts of power, enough to power a small town. The human brain, by contrast, runs on about 20 watts. That’s the power of a dim lightbulb. This is the inspiration behind neuromorphic computing, a new approach to chip design that mimics the structure of the brain.
Instead of the rigid architecture of CPUs and GPUs, neuromorphic chips use “spiking neural networks” that communicate with short bursts of energy, just like neurons. This makes them incredibly energy-efficient. I visited a research lab last year that had a neuromorphic chip the size of a postage stamp running a complex object recognition task for a week on a single coin-cell battery. The implications are enormous.
Imagine a world where every device has its own powerful AI, running for years without needing a recharge. Smart sensors that can monitor a forest for fires, medical implants that can predict a heart attack, and autonomous drones that can navigate for months on end. This is the promise of neuromorphic computing. It’s still early days, but I’m convinced that by 2027, we’ll see the first commercial applications of this technology, and it will fundamentally change edge AI.
3. Optical Computing: Let There Be Light
As AI models get larger, the bottleneck is no longer just the compute, it’s the communication. Moving data between chips and between servers is becoming a major challenge. The copper wires we use today are hitting their physical limits. The solution? Light.
Optical computing uses photons instead of electrons to move data. This allows for much higher bandwidth and lower latency. I’ve seen demos of optical interconnects that can move terabytes of data per second, a 100x improvement over what’s possible today. This will be essential for training the next generation of massive AI models.
But the really exciting part is the potential for all-optical computing, where the computation itself is done with light. This could lead to a new class of AI accelerators that are orders of magnitude faster and more efficient than anything we have today. I have a small, speculative investment in a startup that is building an optical processor for matrix multiplication, a key operation in AI. They’re still in the lab, but if they succeed, they could upend the entire hardware market. It’s a long shot, but the potential payoff is huge.
4. The Disaggregated Data Center: A Computer Made of Lego
The traditional data center is a monolithic beast. Servers are pre-configured with a fixed ratio of CPUs, GPUs, and memory. This is incredibly inefficient. For some workloads, you might need a lot of GPU power but not much memory. For others, the opposite is true. You end up with a lot of stranded resources.
The solution is the disaggregated data center. The idea is to break the server down into its component parts (CPUs, GPUs, memory, storage) and connect them all with a high-speed fabric. This allows you to create “virtual servers” with the exact resources you need for a given workload. It’s like building a computer out of Lego bricks.
This is a huge engineering challenge, but the payoff is enormous. I’m an advisor to a company that is building a disaggregated data center platform, and their early customers are seeing a 30% improvement in resource utilization. That translates directly to cost savings. It also allows for much greater flexibility. As new hardware becomes available, you can simply add it to the pool of resources without having to replace entire servers. This is the future of the AI data center.
5. The Intelligent Edge: AI Everywhere
For the past few years, the focus of AI has been on massive models running in the cloud. But the future of AI is at the edge. From smart cameras and drones to autonomous cars and robots, we’re seeing a proliferation of devices that need to make intelligent decisions in real-time, without relying on a connection to the cloud.
This requires a new class of AI hardware. These chips need to be small, low-power, and able to run complex AI models with low latency. I’m seeing a ton of innovation in this space. Companies are building specialized processors for computer vision, natural language processing, and robotics. One of my portfolio companies is building a chip for autonomous drones that can perform real-time object detection and tracking while consuming less than a watt of power.
This is a massive market opportunity. The number of edge devices is expected to grow to a trillion by the end of the decade. Every one of those devices will need some form of AI hardware. The companies that can build the best edge AI chips will be the NVIDIAs of the next decade.
6. The Analog Comeback: A Blast from the Past
In our digital world, it’s easy to forget that the real world is analog. And for some AI workloads, analog computing can be much more efficient than digital. The idea is to perform computations using continuous physical quantities, like voltages or currents, instead of the discrete 1s and 0s of digital computing.
This approach is particularly well-suited for tasks like image recognition and natural language processing, where there is a lot of noise and uncertainty in the data. I’ve seen a demo of an analog chip that could perform image classification with 100x less power than a digital chip. The tradeoff is that analog computing is less precise than digital. But for many AI applications, you don’t need perfect precision. You just need to be “good enough.”
This is a contrarian bet, but I believe we’re going to see a resurgence of analog computing for specific AI workloads. It won’t replace digital computing, but it will be a powerful tool in the AI hardware toolbox. I’m actively looking for startups in this space. It’s a bit of a wild card, but the potential is too big to ignore.
The Road Ahead
The next five years will be a wild ride for AI hardware. The trends I’ve outlined here are not mutually exclusive. In fact, they will likely reinforce each other. We’ll see custom silicon being used in disaggregated data centers, neuromorphic chips powering the intelligent edge, and optical interconnects connecting them all together.
For entrepreneurs and investors, this is a time of incredible opportunity. The hardware landscape is being redrawn, and new giants will emerge. The key is to understand the fundamental shifts that are happening and to bet on the teams that have the vision and the technical chops to build the future. It’s not going to be easy, but as I learned that day back in 2021, the biggest challenges often lead to the biggest rewards. The race is on. Let’s build.
Frequently Asked Questions
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.