I once watched a founder spend three months and $200,000 building an AI feature that scored a perfect 10 on their RICE spreadsheet. The team was convinced it was the next big thing. When they finally shipped it, the sound was deafening. Crickets. Almost zero adoption. The feature, which was supposed to predict customer churn with 95% accuracy, was technically brilliant but solved a problem nobody actually had. The company nearly went under.
This isn’t a rare story. I’ve seen it happen dozens of times. As an investor in over 200 companies, including AI leaders like Anthropic, OpenAI, and Scale AI, I get a front-row seat to the patterns of success and failure. And one of the most common failure patterns I see is a blind faith in generic prioritization frameworks.
Your RICE score is lying to you. So is your ICE score. These models, born in a different era of software development, are fundamentally broken for the world of AI.
Why Your Framework is Failing You
Most prioritization frameworks like RICE (Reach, Impact, Confidence, Effort) were designed for predictable, non-AI software. They assume you can reasonably estimate each variable. But with AI, that’s a fantasy.
- Reach: You think you know who will use your AI feature, but the most powerful applications are often discovered by users in unexpected ways. Your initial reach estimate is probably wrong.
- Impact: How do you quantify the impact of a slightly more accurate recommendation engine or a text generator that’s 10% more creative? It’s a guess, at best.
- Confidence: This is the biggest lie. Your confidence in your estimates for an AI feature should be, by default, extremely low. The technical uncertainty is massive. The user adoption uncertainty is even bigger. A confidence score of 80% is pure hubris.
- Effort: I’ve seen AI projects quoted at 2 months of effort spiral into 9-month death marches because of unforeseen data challenges or model training issues.
Sticking these guesses into a spreadsheet and multiplying them gives you a false sense of scientific precision. It’s a formula for building impressive-sounding features that don’t move the needle. You end up with a product full of "AI" that feels more like a science fair project than a must-have tool.
The Data That Shocked Me
I didn’t just feel this in my gut. I started tracking it. I looked at 30 of my portfolio companies that were building out their first major AI feature. The results were stark.
Of the 18 companies that used a traditional RICE or ICE framework, only 4 (22%) shipped a feature that achieved its primary business goal within 6 months. The rest either missed the mark completely or had to do a major pivot.
Now, look at the 12 companies that used a different approach—one focused on what I call "Problem Velocity." Of those 12, 9 of them (a full 75%) hit their goals. They moved faster, they learned more, and they built things people actually paid for.
What were they doing differently? They threw out the spreadsheets and focused on one thing: the speed at which they could validate a solution to a painful customer problem.
The Counterintuitive Approach: Problem Velocity
Forget feature prioritization. Start thinking in terms of Problem Velocity. It’s a simple concept: how quickly can you get from identifying a customer’s hair-on-fire problem to putting a potential solution (even a crappy, manual one) in their hands?
This isn’t about building a full-fledged AI model. It’s about faking it. It’s about using humans, simple scripts, or off-the-shelf APIs to simulate the outcome of the AI you plan to build.
At my first company, MovieLaLa, we wanted to build a recommendation engine to suggest movies. Instead of spending six months on collaborative filtering algorithms, we spent a weekend building a simple interface. Behind the scenes, it was just me and my co-founder manually looking at a user’s favorite movies and sending them a list of what we thought they’d like. We faked it. It was a total "Wizard of Oz" setup.
It took us 48 hours and maybe $100 to launch. The "data" we got back was invaluable. We learned what kind of recommendations people clicked on, what they ignored, and how they talked about movies. That feedback was infinitely more valuable than any "Impact" score on a spreadsheet. When we did eventually build the real AI, we knew exactly what to build.
A Step-by-Step Guide for Founders
Ready to try this? Here’s how to implement a Problem Velocity framework. It’s not a formula, it’s a mindset.
Step 1: Find the Pain. Real Pain.
Stop asking customers "what AI features would you like?" They don't know. Instead, ask them about their workflow. Where do they get stuck? What’s the most tedious, annoying, or expensive part of their day? Listen for the emotion. Listen for words like "I hate," "it’s impossible," or "I waste so much time on..."
Your goal is to find a problem so painful they’d be willing to use a clunky, half-broken solution if it made the pain go away.
Step 2: Define the "Magic Wand" Outcome
Once you have the problem, ask the customer: "If you had a magic wand, what would happen?" Don’t let them describe the feature. Force them to describe the outcome.
- Bad: "I want an AI that summarizes my meetings."
- Good: "I want to walk out of a meeting and have a 3-bullet-point summary with action items waiting in my inbox."
See the difference? The second one is a testable outcome. The first is a vague feature idea.
Step 3: Build the "Wizard of Oz" MVP
Now, figure out the absolute fastest, cheapest way to deliver that magic wand outcome. This is where you get creative.
- Can you do it manually behind the scenes?
- Can you use a simple Zapier integration and a Google Sheet?
- Can you just use the OpenAI API playground with a pre-written prompt and have a human copy-paste the result?
At RemoteTeam (which was acquired by Gusto), we wanted to automate parts of international compliance. Building the real system was a massive legal and technical undertaking. But our first version? It was a simple form that emailed our team, and we would manually research the answer and email it back. It wasn’t scalable, but it proved people would pay for the outcome.
Step 4: Get It In Front of 5 Customers
Don’t try to launch this to everyone. Find five, and only five, friendly customers who feel the pain intensely. Give them the "Wizard of Oz" solution. Watch them use it. Sit with them. Does it actually solve the problem? Is the outcome what they expected? Is the magic real?
The feedback you get here is pure gold. It’s not hypothetical. It’s based on a real (simulated) experience. This is where you learn. You might find out the problem you thought you were solving isn’t the real one. Or you might find that a much simpler solution is good enough.
This is How You Win
Building an AI product isn’t about having the most sophisticated model. It’s about having the tightest loop between a customer problem and a validated solution. The companies that win are the ones that can learn the fastest.
Throwing out your spreadsheets feels scary. It feels like you’re losing control and rigor. But what you’re really doing is trading the illusion of certainty for the reality of speed and learning. Stop prioritizing features and start accelerating your time to customer value.
That’s it. That’s the secret. Stop playing with formulas and go talk to your customers. Find their pain and build a Wizard of Oz test this weekend. The data you get will be worth more than a thousand RICE scores.
Frequently Asked Questions
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.