Evaluating the technical claims of an AI startup is a critical step for any investor. It involves a combination of scrutinizing the team's expertise, testing the product's capabilities, and understanding the underlying data and algorithms to separate genuine innovation from hype.
As an angel investor with a portfolio of over 50 startups and the co-founder of an AI company, I've seen my fair share of pitches. The rise of AI has brought a wave of innovation, but it has also created a fog of buzzwords and exaggerated claims. For investors, especially those without a deep technical background, performing technical due diligence can feel like a daunting task. However, with the right framework, you can effectively assess the viability and defensibility of an AI startup's technology. This guide will walk you through the key steps for conducting a thorough AI evaluation.
1. Assess the Team's Technical Depth
Before you even look at the product, look at the people building it. An AI product is only as strong as the team behind it. You need to be convinced that they have the expertise to not only build the current version but also to innovate and overcome future technical challenges.
- Founder's Background: Do the founders have a history in AI, machine learning, or a related field? A PhD from a top university is great, but practical experience shipping AI products is even better.
- Engineering Team: Look at the profiles of their key engineers. Have they worked on scalable AI systems before? What are their areas of expertise?
- Advisors: A strong technical advisory board can be a good sign, but be wary of advisors who are just names on a slide. Ask how actively they are involved.
2. Deconstruct the "AI Magic"
Many startups describe their technology as a black box powered by "proprietary AI." Your job is to open that box. You don't need to be a machine learning engineer, but you do need to ask the right questions to understand what's actually happening under the hood.
Here is a step-by-step process to guide your inquiry:
- Problem-Solution Fit: Start by understanding the problem they are solving. Is it a problem that genuinely requires an AI solution, or could it be solved with simpler, more traditional software? The best AI investing opportunities are in startups that use AI to solve a problem in a way that was not possible before.
- Algorithm and Model: Ask about the specific algorithms and models they are using. Are they using well-established models like GPT-3 or Stable Diffusion, or have they developed something truly novel? If it's proprietary, they should be able to explain why their model is better without revealing the "secret sauce."
- Data as a Moat: Data is the lifeblood of AI. Understand where their training data comes from. Do they have a unique, proprietary dataset? How are they continuing to acquire data to improve their models over time? A startup with a strong data acquisition strategy has a significant competitive advantage.
Pro Tip: Ask the founders to walk you through a live demo and explain the technical details in real-time. Pay attention to how they handle unexpected inputs or edge cases. A robust system will handle these gracefully, while a brittle one may break.
3. Validate the Product's Performance
Technical claims are meaningless without real-world performance. You need to see the product in action and, if possible, test it yourself. This is where you separate the impressive demos from the functional products.
- Live Demo: The demo should be live, not a pre-recorded video. Ask them to use the product to solve a real problem, perhaps one from your own experience.
- Customer Case Studies: Talk to their existing customers. How are they using the product? What has been their experience with its performance and reliability? For more on this, see my article on how to conduct customer interviews.
- Pilot or Trial: The gold standard of validation is to use the product yourself. If possible, negotiate a pilot or trial period to get hands-on experience. This will give you the most accurate picture of its capabilities and limitations.
4. Analyze the Technical Architecture and Scalability
A great AI model is not enough. The startup needs a robust and scalable technical architecture to deliver it as a reliable service. This is especially important for B2B SaaS companies that need to serve a large number of users.
- Infrastructure: Where is the product hosted? Are they using a major cloud provider like AWS, Google Cloud, or Azure? This is a good sign for scalability and reliability.
- Tech Stack: What technologies are they using to build their application? While there's no single "right" stack, it should be a modern, well-supported set of technologies.
- Scalability Plan: Ask the team how they plan to scale the system as their user base grows. They should have a clear roadmap for handling increased load and data volume. For early-stage companies, this might not be fully built out, but the founding team should demonstrate a clear understanding of the architectural principles required to scale, a topic I cover in more detail in scaling for growth.
Key Takeaway: A brilliant AI model is useless if it can't be delivered reliably to users. Don't underestimate the importance of a solid, scalable infrastructure. It's often the difference between a cool demo and a viable business.
5. Understand the Competitive Landscape
Finally, no technical evaluation is complete without an understanding of the competitive area. How does this startup's technology compare to its competitors? For a deeper dive into competitive analysis, you can read my thoughts on building a competitive moat.
- Direct Competitors: Who are the other startups trying to solve the same problem? How does their technology differ?
- Incumbents: Are there large, established companies in this space? They may have more resources and data, but they may also be slower to innovate.
- Defensibility: What is the startup's long-term defensible advantage? Is it their proprietary data, their unique algorithm, or their deep domain expertise? A strong technical moat is essential for long-term success in the world of AI investing.
By following these steps, you can move beyond the buzzwords and gain a true understanding of an AI startup's technical claims. It requires diligence and a willingness to ask tough questions, but it's the only way to make informed investment decisions in this exciting and rapidly evolving space.
Conclusion
Investing in AI startups presents a tremendous opportunity, but it demands a rigorous approach to due diligence. By systematically evaluating the team, deconstructing the technology, validating performance, and analyzing the architecture and competitive world, you can build the confidence needed to back the next generation of transformative companies. Remember that genuine innovation is more than just a clever algorithm; it's about a deep understanding of a problem and a scalable, defensible solution. The frameworks discussed here are the same ones I apply when considering an investment, helping me deal with the complexities of AI evaluation and identify the startups with true potential.
Frequently Asked Questions
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.