I almost burned out. Not from the long hours, the endless meetings, or the pressure of shipping products to millions of users. I almost burned out from chasing ghosts.
For years, I was obsessed with the wrong numbers. I led AI product teams at Google and Amazon, and I was surrounded by some of the smartest people in the world. We were all chasing the same thing: perfection. We wanted our models to have the highest accuracy, the lowest error rate, and the most impressive-looking metrics. We were so focused on the numbers that we lost sight of what really mattered: the user.
I remember one project in particular. We were building a new recommendation engine, and we were determined to make it the best in the world. We spent months tweaking our algorithms, running endless A/B tests, and celebrating every incremental improvement in our offline metrics. We were convinced we were on the right track. But when we finally launched the product, the results were… underwhelming. Users weren _t engaging with the new recommendations. They weren’t clicking, they weren’t buying, and they certainly weren’t telling their friends about it.
We were crushed. We had poured our hearts and souls into this project, and it was a flop. We had all the data in the world, but we had failed to understand our users. We were so focused on the “what” that we had forgotten about the “why.”
The Allure of Flawed Metrics
It’s easy to fall into the trap of chasing flawed metrics. They’re seductive. They’re easy to measure, they’re easy to report, and they give you a false sense of progress. But they’re also dangerous. They can lead you down the wrong path, and they can blind you to what’s really important.
In the world of AI, the most common flawed metrics are accuracy, precision, and recall. These are the metrics that data scientists love to talk about. They’re the metrics that get you published in academic papers, and they’re the metrics that win you Kaggle competitions. But they’re not the metrics that build great products.
Why? Because they don’t measure what really matters: user outcomes. They don’t tell you if your users are happy, if they’re engaged, or if they’re getting value from your product. They’re just numbers on a spreadsheet.
I learned this the hard way. I spent years chasing these flawed metrics, and it almost cost me my career. I was so focused on the numbers that I lost sight of the big picture. I was so obsessed with perfection that I forgot about the people I was building for.
My "Aha!" Moment
My “Aha!” moment came during a late-night meeting with my team. We were reviewing the latest A/B test results, and the numbers were not good. Our new recommendation engine was performing worse than the old one, and we couldn’t figure out why.
We had tried everything. We had tweaked our algorithms, we had added new features, and we had even tried to bribe users with discounts. Nothing worked.
I was at my wit’s end. I was about to give up when one of our junior engineers spoke up. “What if we’re measuring the wrong thing?” she asked.
It was a simple question, but it hit me like a ton of bricks. She was right. We were so focused on the numbers that we had forgotten to ask ourselves the most important question: what problem are we trying to solve for our users?
That question changed everything. We went back to the drawing board, and we started from scratch. We talked to our users, we listened to their feedback, and we tried to understand their needs. We stopped worrying about the numbers, and we started focusing on the people.
And you know what? It worked. We launched a new version of our recommendation engine, and it was a huge success. Users loved it, and they couldn’t get enough of it. We had finally found the right metric: user happiness.
The Framework I Use Now
After that experience, I developed a new framework for building AI products. It’s a simple framework, but it’s incredibly powerful. It’s based on three core principles:
- Start with the user. Before you write a single line of code, you need to understand your users. What are their needs? What are their pain points? What are they trying to accomplish? Once you understand your users, you can start to build a product that they’ll love.
- Focus on outcomes, not outputs. Don’t get bogged down in the details of your algorithms. Instead, focus on the outcomes you’re trying to achieve. Are you trying to increase user engagement? Are you trying to drive revenue? Are you trying to make your users’ lives easier? Once you know what you’re trying to achieve, you can start to measure the right things.
- Iterate, iterate, iterate. Don’t be afraid to fail. The best AI products are built through a process of trial and error. Launch your product, get feedback from your users, and then iterate. The faster you can iterate, the faster you’ll learn, and the faster you’ll build a great product.
This framework has served me well over the years. I’ve used it to launch AI products that have reached millions of users, and I’ve used it to build two successful companies. It’s not a magic bullet, but it’s a start. It’s a way to avoid the mistakes I made, and it’s a way to build AI products that people love.
A Real-World Example
At RemoteTeam, we were building a product to help remote teams stay connected. We had all sorts of ideas for features we could build. We could build a virtual water cooler, we could build a tool for remote team-building activities, or we could build a platform for virtual offsites.
Instead of just picking an idea and running with it, we started with the user. We interviewed dozens of remote team managers, and we asked them about their biggest challenges. We learned that their biggest pain point was not a lack of tools, but a lack of connection. They felt isolated, and they missed the camaraderie of the office.
That insight was a game-changer. We realized that we didn’t need to build a bunch of new features. We just needed to build a product that would help remote teams feel more connected. We ended up building a simple Slack bot that would randomly pair up team members for virtual coffee chats. It was a simple idea, but it was a huge success. It helped our users feel more connected, and it helped us build a thriving business.
My Investment Philosophy: A Lesson from AI's Best
My experiences building AI products didn't just shape me as a product leader; they fundamentally changed my approach to angel investing. I’ve been fortunate to invest in over 200 companies, including some of the most transformative AI companies of our time like Anthropic, OpenAI, Scale AI, and Hugging Face. My investment thesis is simple: I invest in people who are obsessed with solving real user problems, not just chasing technological novelty.
When I meet founders, I don't want to see a deck full of vanity metrics. I don't care about their model's accuracy to the fifth decimal place. I want to know what user pain point they are solving. I want to know how they are measuring user love. I want to see that they have a deep, almost obsessive, understanding of their users.
The founders of the most successful AI companies I've invested in all have this in common. They are not just brilliant technologists; they are product visionaries. They understand that the best AI is invisible. It's the AI that seamlessly integrates into a user's workflow and makes their life easier without them even realizing it. It's the AI that feels like magic.
This is the lesson I learned the hard way, and it's the lesson I try to impart to every founder I meet. Don't chase the ghosts of flawed metrics. Chase the real, tangible impact you can have on your users' lives. That's where you'll find true product-market fit, and that's where you'll build a company that lasts.
Don't Fall for Vanity Metrics
It's so easy to get seduced by vanity metrics. These are the numbers that look good on a slide deck but don't actually mean anything for your business. Things like page views, download numbers, or even just 'active users' without any context. They are the empty calories of the analytics world.
I've seen so many startups fail because they were chasing vanity metrics. They would celebrate hitting 100,000 downloads, but they wouldn't have a clue how many of those users were actually using the app. They would boast about their millions of page views, but they couldn't tell you if any of those views were turning into customers.
Don't be that startup. Focus on the metrics that matter. Focus on the metrics that tell you if your users are getting value from your product. Are they coming back every day? Are they telling their friends about you? Are they paying you for your product? These are the metrics that will tell you if you're on the right track.
The Bottom Line
Building great AI products is not about having the best algorithms or the most impressive-looking metrics. It's about understanding your users and solving their problems. It's about focusing on outcomes, not outputs. And it's about iterating until you get it right.
I almost burned out chasing the wrong things. I was so focused on the numbers that I lost sight of what really mattered. Don't make the same mistake I did. Start with the user, focus on outcomes, and never stop iterating. If you do that, you'll be well on your way to building AI products that people love.
And more importantly, you'll be building a business that lasts.
Frequently Asked Questions
What tools do I need to get started?
Start with the basics. You don't need expensive software or fancy tools. A spreadsheet, a note-taking app, and direct access to your customers will get you further than any enterprise platform. Add tools only when you hit a specific bottleneck.
How do I measure success with this approach?
Pick one or two metrics that directly tie to your goal and track them weekly. Vanity metrics like page views or follower counts rarely matter. Focus on metrics that reflect real engagement or revenue impact.
Do I need technical skills to almost burned out chasing flawed ai metrics (and what i do now)?
Not necessarily. While technical understanding helps, the most important skills are clear thinking and the ability to break problems into smaller pieces. Many successful founders I've invested in started with zero technical background and either learned enough to be dangerous or found the right technical partner.
How long does it take to almost burned out chasing flawed ai metrics (and what i do now)?
The timeline varies depending on your starting point and resources. For most founders, expect 2-4 weeks for initial setup and 2-3 months to see meaningful results. I've seen teams move faster when they focus on one thing at a time rather than trying to do everything at once.