I remember the exact moment I realized we were in deep trouble. It was 2 a.m. on a Tuesday, and I was staring at a server rack that was supposed to be the backbone of our new AI venture. The fans were screaming, the room was hot enough to bake bread, and our models were crashing every fifteen minutes. We had raised millions, hired the smartest people I knew, and had a product that could genuinely change an industry. But in that moment, none of it mattered. We were at the mercy of the hardware, and the hardware was failing us.
Everyone is obsessed with AI models. They talk about parameters, architectures, and training data. But almost no one talks about the brutal, physical reality of the machines that run them. After two successful exits and over 200 angel investments in companies like Anthropic and OpenAI, I’ve seen this story play out more times than I can count. I’ve seen brilliant teams with groundbreaking ideas get stopped dead in their tracks, not by a flawed algorithm, but by a shortage of silicon. This is the unfiltered truth about what it really takes to build and scale AI, and it’s a story that starts not in the cloud, but in the trenches with the hardware itself.
The Great GPU Lie
For the last few years, the prevailing wisdom has been to just throw everything into the cloud. Need more compute? Just spin up another instance. It sounds easy, and for a while, it was. But then the AI explosion happened, and suddenly the cloud wasn’t the infinite resource we thought it was. It became the bottleneck. The public cloud providers are in a street fight for every GPU they can get their hands on, and the costs are passed right down to you, the founder.
I was advising a startup in the drug discovery space, a team with some of the brightest minds from Stanford. Their models were complex, requiring massive amounts of processing power to simulate molecular interactions. They were burning through $100,000 a month on cloud credits, a figure that was set to triple as they scaled. Then the GPU shortage hit its peak. Their provider couldn’t guarantee them the instances they needed. Their entire R&D pipeline ground to a halt for three weeks. Three weeks. In the startup world, that’s an eternity. They almost lost a major pharma partnership because of it. That’s not a sustainable way to build a business.
This isn’t a unique story. I’ve seen it happen in autonomous driving, in robotics, in generative art. The cloud is a great place to start, to experiment, to find product-market fit. But it is not the promised land for scaling a serious AI company. Relying on it completely is like building your house on rented land, and the landlord can raise the rent—or kick you out—at any time.
The Custom Silicon Revelation
This brings me to a decision that many in my circle thought was insane at the time: building our own custom silicon. At RemoteTeam, we were processing vast amounts of data to help companies manage their distributed workforces. We needed to run our AI models efficiently and at a low cost. The cloud was eating our margins alive. We looked at the problem and realized that 90% of our compute load was from a few very specific operations. The general-purpose GPUs we were renting were overkill, like using a sledgehammer to crack a nut.
So we took a leap. We hired a small team of chip designers and started building our own ASIC (Application-Specific Integrated Circuit). It was one of the hardest things I’ve ever done. It was a multi-year, multi-million dollar bet. There were moments I was sure we had made a catastrophic mistake. But when we finally got the first wafers back from the foundry and plugged them into our servers, the result was staggering. Our processing costs for our core AI features dropped by over 80%. The performance was an order of magnitude better than what we were getting from the top-of-the-line GPUs. We had turned our biggest cost center into a competitive advantage.
This is a path that more and more serious AI companies are taking. Look at Google with their TPUs, Amazon with Inferentia and Trainium, and Tesla with Dojo. They all realized that to push the boundaries of AI, they couldn't just be software companies. They had to become hardware companies. It’s not for everyone, but for those who are serious about operating at scale, it’s the only move that makes sense.
The Future is on the Edge
The cloud isn’t going away, but its role is changing. It will be the place where we train our massive, foundational models. But the future of AI application, the part that touches the user, is not in a massive, centralized data center. It’s at the edge.
Edge AI means running AI models directly on the device—on your phone, in your car, on a factory floor. This is a fundamental shift. It’s driven by the need for lower latency, better privacy, and lower cost. When a self-driving car needs to make a split-second decision, it can’t wait for a round trip to a data center hundreds of miles away. When you’re using a real-time translation app, you don’t want your private conversations being sent to the cloud.
I’m putting my money where my mouth is. A significant portion of my recent investments are in companies building the hardware and software for the edge. Think low-power chips that can run sophisticated models on a tiny battery, or new types of sensors that can process data locally. This is where the next wave of innovation will come from. It’s a tougher, more fragmented market than the cloud, but it’s also where the real-world impact will be felt.
And what about the next frontier? I’m keeping a close eye on quantum computing. It’s still early, and there’s a lot of hype, but the potential is undeniable. For certain classes of problems, particularly in materials science and drug discovery, quantum computers could provide a leap in computational power that makes today’s supercomputers look like pocket calculators. It’s a long-term bet, but in the world of deep tech, you have to be skating to where the puck is going to be, not where it has been.
My Advice for Founders
So, what does this all mean for you, the founder trying to build the next great AI company? Here are my unfiltered thoughts:
- Don’t be seduced by the cloud. Use it as a tool, but don’t let it become a crutch. Understand your cost structure from day one and have a plan to move off it as you scale.
- Think about your hardware strategy from the beginning. You don’t need to build your own chip on day one, but you do need to understand the hardware your models will run on. Optimize your code for specific hardware. Squeeze every drop of performance out of the metal.
- Embrace the edge. If your application can run on the edge, it probably should. The advantages in latency, privacy, and cost are too significant to ignore.
- Be contrarian. The biggest opportunities are often in the places no one else is looking. While everyone is chasing the next big language model, look at the hardware that will run it. While everyone is building in the cloud, look to the edge.
Building an AI company is brutally hard. You have to get the model, the data, and the product right. But as I learned that long night in the server room, you also have to get the hardware right. It’s the foundation upon which everything else is built. And in the coming years, the companies that understand this, the ones that master the full stack from the silicon to the software, are the ones that will win.
A Deeper Dive into the Silicon Bet
When I say building our own chip was hard, I'm not just talking about the financial risk. It was a logistical and technical nightmare. We had to recruit a team of world-class engineers who were willing to take a bet on a startup over established players like NVIDIA or Intel. We spent months just developing the architecture, running simulations day and night. There was a constant fear that we'd miscalculate something, that we'd spend millions on a piece of silicon that was fundamentally flawed. I remember our lead engineer, a brilliant but notoriously pessimistic guy named Ken, coming into my office with a stack of printouts, his face pale. He thought he had found a critical timing bug in the memory controller. It would have set us back six months and cost us another million dollars in mask revisions. We spent the next 72 hours locked in a conference room, fueled by stale coffee and pizza, poring over every line of Verilog. It turned out to be a false alarm, a bug in the simulation software itself, not our design. The relief was immense, but it was a stark reminder of how many things could go wrong. That's the reality of building hardware. It's not like software where you can just push a patch. Mistakes are measured in millions of dollars and months of delays.
The Edge is Sharper Than You Think
Let's talk more about the edge. It's not just about your phone or your car. Think about a modern hospital. You have hundreds of smart medical devices – infusion pumps, patient monitors, diagnostic imagers. Historically, they've been dumb terminals. Now, we can embed intelligence directly into them. A smart infusion pump can use a tiny camera to verify the drug and dosage against the patient's electronic health record, preventing catastrophic errors. A patient monitor can use AI to predict a cardiac event minutes before it happens, alerting the nursing staff. None of this is practical if you're relying on a slow, unreliable Wi-Fi connection to a distant cloud server. The computation has to happen right there, right now. That's the power of the edge. It's about taking AI out of the data center and putting it into the real world, where it can make a tangible difference in people's lives.
Another area I'm incredibly excited about is industrial robotics. I visited a factory recently that was using AI-powered robots for quality control. These robots had high-resolution cameras and were running complex computer vision models to detect microscopic defects in circuit boards. The entire process was happening in real-time, on the factory floor. The robots were learning and adapting, getting better with every board they inspected. This is only possible with powerful edge computing. The sheer volume of data being generated by those cameras would be impossible to stream to the cloud for processing. By processing it locally, they were able to achieve a level of quality and efficiency that was simply unattainable before.
This is the future I'm betting on. A future where intelligence is distributed, where every device has the potential to be smart, and where the cloud is just one piece of a much larger, more complex puzzle. The road to get there is paved with hardware, and for the founders who are willing to get their hands dirty and build the picks and shovels of this new gold rush, the rewards will be immense.
Frequently Asked Questions
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.