Your AI startup is probably going to die.
There, I said it. Everyone is obsessed with the latest AI models. GPT-4, Claude 3, Llama 3. It's a constant hype cycle. But here's the dirty little secret nobody talks about: your model is only as good as the infrastructure it runs on. After two exits and over 200 angel investments in companies like Anthropic, OpenAI, and Scale AI, I've seen more startups die from infrastructure mistakes than from bad models. I'm here to share the unfiltered truth about what it really takes to build and scale AI infrastructure.
1. Blindly Following the NVIDIA Hype
The entire industry has a single point of failure, and its name is NVIDIA. The GPU shortage is real, and it's brutal. I remember back at RemoteTeam, we were scrambling to get our hands on enough GPUs to train our models. We were paying 3x the list price on the spot market, and it was still a nightmare. We were so desperate we even considered driving a truck to a data center in the middle of nowhere and... let's just say we got creative. Relying on a single vendor is a recipe for disaster. You're at the mercy of their supply chain, their pricing, and their roadmap. Don't be a lemming. Explore alternatives. Look at AMD, look at Intel, look at the cloud providers. Diversify your hardware stack before it's too late.
2. Ignoring Custom Silicon (TPUs and beyond)
Google's TPUs are their secret weapon. They are custom-built for machine learning, and they are incredibly efficient. While the rest of the world is fighting for NVIDIA's scraps, Google is quietly building a massive competitive advantage with their own silicon. And they're not the only ones. Amazon has Trainium and Inferentia. Microsoft has Maia. The future of AI is custom silicon. As a founder, you need to be thinking about this. You might not be able to build your own chip, but you can and should be using the custom silicon that's available to you in the cloud. I have an investment in a company called Spectra Computing that is designing custom chips for a very specific AI workload, and they are seeing 10x performance improvements over off-the-shelf GPUs. That's the kind of advantage that can make or break a startup.
3. Underestimating the Power of the AI Cloud
I get it. You want to own your own hardware. You want to have full control over your infrastructure. But for most startups, that's a huge mistake. The AI cloud is your best friend. It's flexible, it's scalable, and it's cost-effective. I had a portfolio company that was trying to build out their own data center. They spent months and millions of dollars, and they were still nowhere near ready to launch. I finally convinced them to move to the cloud, and they were up and running in a matter of weeks. They went from a $2 million-a-year burn rate on hardware to a $500k bill on AWS, and they could scale to the moon. Don't let your ego get in the way of making the right decision. Use the cloud.
4. Getting Seduced by the "Quantum" Dream
Quantum computing is a siren song for founders who love shiny objects. It's a fantasy. I see so many founders getting distracted by the quantum dream. They're trying to build quantum machine learning models, but they can't even get their classical models to work. It's a massive waste of time and money. For 99% of startups, quantum computing is a distraction. Focus on the here and now. Focus on building a great product with the technology that's available today. There will be a time for quantum, but it's not now.
5. Building for Today, Not for Tomorrow
This is the most common mistake I see. Founders are so focused on getting their MVP out the door that they don't think about the long-term implications of their infrastructure choices. I learned this lesson the hard way at MovieLaLa. We built our entire infrastructure on a technology that was hot at the time, but it didn't scale. We had to re-architect everything, and it was a painful and expensive process. When you're making infrastructure decisions, you need to be thinking about where your company is going to be in 2, 5, or even 10 years. Don't build yourself into a corner. Build for the future.
The Bottom Line
Stop obsessing over models and start paying attention to your infrastructure. It's not as sexy, but it's just as important. The choices you make today will have a huge impact on your company's future. So be thoughtful, be strategic, and don't be afraid to go against the grain. Your startup's life depends on it.
Frequently Asked Questions
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.