I’m probably going to get a lot of hate for this, but it needs to be said: your approach to building AI products is fundamentally flawed. We're all chasing shiny AI objects and forgetting the first principles of building great products. Here's the unpopular opinion that might just save your startup.
I’ve seen it more times than I can count. I’ve sat in boardrooms, listened to pitches from founders, and reviewed countless product roadmaps. They all have one thing in common: a deep, unwavering faith in the power of AI to solve all their problems. They talk about their grand vision for a fully autonomous, self-learning, paradigm-shifting AI that will revolutionize their industry. They show me beautiful Gantt charts with neat little boxes for "Build the model," "Deploy the API," and "Achieve 99% accuracy."
And most of the time, it’s complete bullshit.
It’s a fantasy. A beautifully crafted, well-intentioned fantasy that is completely disconnected from the messy reality of building a successful product. The truth is, your AI roadmap is built on a mountain of unexamined assumptions. And those assumptions are a ticking time bomb that will blow up your budget, your timeline, and your company if you don’t defuse them.
I learned this the hard way. Early in my career, we were building a feature at one of my startups. We were convinced that a sophisticated recommendation engine would be a game-changer. We spent six months and a significant chunk of our seed funding building it. We hired expensive data scientists, built a complex data pipeline, and trained a model that we were incredibly proud of. When we finally launched it, the reaction was… a resounding ‘meh’. Nobody used it. All that time, all that money, all that effort—gone. Why? Because we never stopped to ask the most basic question: "Do our users even want AI-powered recommendations?" We just assumed they did.
That failure was a painful but necessary lesson. It forced me to develop a new way of thinking about product development, especially for AI. I call it the Assumption-Testing Framework. It’s not sexy. It’s not about "disruption" or "10x thinking." It’s about systematically de-risking your AI roadmap by treating every single part of it as a hypothesis, not a fact.
The Core Idea: Stop Building, Start Testing
The framework is simple. Instead of building a full-featured AI product, you identify the biggest, riskiest assumptions in your plan and find the cheapest, fastest way to test them. The goal is to get real-world feedback as quickly as possible, so you can either validate your assumptions and proceed with confidence, or invalidate them and pivot before you’ve wasted millions of dollars.
Think of it like a scientist in a lab. You don’t just mix a bunch of chemicals together and hope for the best. You form a hypothesis, design an experiment to test it, and then analyze the results. Building an AI product should be no different.
Here’s how you can apply the Assumption-Testing Framework to your own AI roadmap.
Step 1: Uncover Your Hidden Assumptions
First, you need to dig deep and identify all the assumptions that your AI roadmap is built on. Get your team in a room and start asking the hard questions. What has to be true for this product to succeed? Don’t hold back. The more assumptions you can uncover, the better.
Here are some common categories of assumptions I see all the time:
- Problem/Solution Assumptions: Do users actually have the problem you think they have? Is your AI-powered solution the right way to solve it? Will they be willing to pay for it?
- Data Assumptions: Do you have access to the data you need to train your model? Is the data high-quality? Is it legally and ethically okay to use it? I’ve seen companies spend a year building a data pipeline only to realize the data they were collecting was useless.
- Technical Feasibility Assumptions: Can you actually build the AI model you’re envisioning? Can you achieve the level of accuracy you need for the product to be useful? Is it possible to run the model at a reasonable cost and speed?
- User Behavior Assumptions: Will users trust your AI? Will they be willing to change their workflow to use your product? Will they understand how to interact with it?
Write every single assumption down on a whiteboard or in a shared document. Be brutally honest with yourselves. The more you can uncover now, the less pain you’ll experience later.
Step 2: Prioritize Your Assumptions by Risk
Once you have a long list of assumptions, you need to figure out which ones to test first. Not all assumptions are created equal. Some are much riskier than others. A simple way to prioritize is to use a 2x2 matrix. On one axis, you have the impact of the assumption being wrong (from low to high). On the other axis, you have your confidence in the assumption being true (from low to high).
Your highest-priority assumptions are the ones in the High Impact / Low Confidence quadrant. These are the assumptions that, if they turn out to be false, will completely derail your project. And they’re the ones you have the least evidence for.
For example, let’s say you’re building an AI-powered tool that helps lawyers review contracts. One of your assumptions might be: "We can achieve 99.9% accuracy in identifying risky clauses." The impact of this being wrong is massive. If your tool misses a critical clause, your customer could get sued for millions of dollars. Your confidence in this assumption is probably low, because achieving that level of accuracy is incredibly difficult. This is a classic High Impact / Low Confidence assumption, and it’s the first one you should test.
Step 3: Design the Cheapest, Fastest Test
Now comes the fun part. For each of your high-priority assumptions, you need to design an experiment to test it. The key here is to be creative and scrappy. The goal is not to build a perfect, scalable solution. The goal is to get a signal from the real world as quickly and cheaply as possible.
Here are some of my favorite techniques for testing AI assumptions:
The "Wizard of Oz" Test: This is my go-to for testing the value proposition of an AI product. Instead of building a complex AI model, you have a human perform the task behind the scenes. The user thinks they’re interacting with a sophisticated AI, but it’s really just a person in a chat window or an Excel spreadsheet. This is a fantastic way to test whether users actually want your AI-powered solution before you write a single line of code. We did this at RemoteTeam to test a new automated payroll feature. It was just me and a spreadsheet for the first three months, manually running payroll for our beta customers. It was a ton of work, but it proved that customers were willing to pay for the feature, which gave us the confidence to go and build the real thing.
The "Fake Door" Test: This is a great way to test demand for a new feature. You simply add a button or a link for the feature in your UI. When a user clicks on it, you show them a message that says "Coming Soon!" and maybe ask them to sign up for a waitlist. The number of people who click on the button is a powerful signal of how much demand there is for the feature. It’s a simple, low-effort way to get quantitative data on user intent.
The Data Feasibility Spike: Before you commit to building a massive data pipeline, take a small, representative sample of your data and try to manually clean it and train a simple model. This "spike" will give you a much better sense of the challenges you’ll face when you try to do it at scale. You might discover that the data is much messier than you thought, or that it’s missing critical information. This is a lesson I’ve seen many of my portfolio companies learn the hard way.
Step 4: Learn and Iterate
The final step is to analyze the results of your experiment and decide what to do next. Did you validate your assumption? Great! You can now proceed with a little more confidence. Did you invalidate your assumption? Also great! You just saved yourself a ton of time and money. Now you can pivot your strategy based on what you’ve learned.
The key is to be intellectually honest. Don’t try to spin the results to fit your preconceived notions. The whole point of this process is to confront the brutal reality of the market. If the data is telling you that you’re on the wrong track, you need to listen.
This is a continuous cycle. You test an assumption, you learn something, you update your roadmap, and then you move on to the next riskiest assumption. It’s a much more humble, iterative way of building products. It’s less about having a grand, top-down vision and more about discovering the right path through a series of small, calculated bets.
Stop Dreaming, Start De-Risking
Look, I get it. It’s fun to dream about the future of AI. It’s exciting to think about all the amazing things we can build. But as entrepreneurs, our job is not to dream. Our job is to build a sustainable business. And the only way to do that is to be relentlessly focused on de-risking our plans.
So I’m begging you, take a hard look at your AI roadmap. I guarantee you it’s full of unexamined assumptions. Your job is to find them, test them, and systematically eliminate them. It’s not as glamorous as building a world-changing AI, but it’s the only way you’re going to survive.
Stop building. Start testing. Your startup depends on it.
Frequently Asked Questions
What if I disagree with some of the advice?
Good. That means you're thinking critically, which is exactly what a good founder should do. Take what resonates, test it, and discard what doesn't work for your specific situation. No advice is universal.
How often is this guide updated?
I revisit and update my guides regularly as I learn new things and as the market evolves. The core principles tend to stay stable, but specific tactics and tools get refreshed based on what's working right now.
Who is this guide designed for?
This guide is written for founders and operators who want practical, actionable advice rather than theoretical frameworks. Whether you're just starting out or scaling an existing business, the principles here apply across stages.