I once burned $2 million chasing a ghost. The ghost was a beautiful, seductive mirage of rising user engagement, painted in vibrant colors on a slick AI-powered dashboard. Every morning, I’d log in and see the numbers climbing up and to the right. It felt like winning. The problem? It was all a lie.
That was at my second startup. We had what I thought was a revolutionary product, and the dashboard confirmed my bias. It told me users were engaged, that they were spending more time on the platform, that our features were a hit. So we doubled down. We poured money into marketing, hired more engineers, and pushed for faster growth, all based on the glowing reports from our analytics tool. The crash came six months later. Revenue was flat, churn was climbing, and the engagement we were so proud of was meaningless. Users were clicking around, sure, but they weren’t converting. They weren’t sticking. The dashboard was a vanity mirror, reflecting what we wanted to see, not the hard truth of a failing business model.
That $2 million lesson was brutal, but it taught me something fundamental: most AI analytics tools are designed to impress, not inform. They sell a fantasy of clarity and control, but in reality, they often obscure the signals that truly matter. After three years, five startups, and analyzing data from over 180 of my portfolio companies, I’ve seen this pattern repeat itself again and again. I’ve seen founders celebrate hockey-stick growth in one metric while the business slowly bleeds out from a thousand tiny cuts hidden from view.
I’m not anti-data. I’m the opposite. I’m obsessed with it. But I’m against the lazy, superficial analysis that these dashboards encourage. I’m here to tell you that the path to real, predictive insights isn’t found in a pre-packaged solution. It’s built, brick by painful brick, from the ground up.
The Seductive Lie of the AI Dashboard
Let’s be honest, the appeal of an AI dashboard is powerful. It promises to turn the chaotic mess of raw data into a clean, intuitive interface. With a few clicks, you get charts, graphs, and a "health score" for your business. It feels like you have a command center for your company, a god’s-eye view of everything that’s happening.
But here’s the first hard truth: the dashboard is not the territory. What you’re seeing is a heavily processed, filtered, and often biased interpretation of reality. The algorithms that power these tools are designed to find patterns, but they don’t understand context. They can’t tell you why a number is going up, or whether that increase is actually good for your business in the long run.
At RemoteTeam, which was later acquired by Gusto, we almost fell into this trap. Our initial dashboard showed a massive spike in daily active users. The team was ecstatic. We were ready to issue a press release. But when we dug into the raw data, we found the spike was caused by a bug in our login system that was creating duplicate user sessions. The AI saw a pattern and reported it as "growth." It had no way of knowing it was a technical glitch. If we had trusted the dashboard, we would have made a fool of ourselves.
This is the core problem. These tools are black boxes. You feed data in, and you get a pretty picture out, with little to no visibility into the logic that produced it. You’re forced to trust that the AI knows what it’s doing, that it understands your business model, your users, and your goals. That’s a dangerous assumption to make.
Vanity Metrics: The Junk Food of Analytics
Most AI dashboards are optimized to show you what I call "vanity metrics." These are numbers that look good on a slide deck but don’t correlate with business success. Think page views, time on site, or number of features used. They’re easy to measure and easy to increase, which is why they’re so popular. They give you a sense of progress without requiring you to do the hard work of actually creating value.
I once advised a startup that was obsessed with its "feature adoption" rate. Their dashboard showed that users were trying out every new feature they released. They saw this as a sign of a healthy, engaged user base. But their revenue was stagnant. When we looked closer, we found that users were trying the features once and then abandoning them. The high adoption rate was actually a symptom of a confusing, poorly designed product. Users were desperately clicking around, trying to find something that worked.
The dashboard didn’t tell them that. It just showed them a big, happy number. It was feeding them junk food, and they were getting fat and slow while their competitors were getting lean and fast.
The antidote to vanity metrics is to focus on what I call "needle-moving" metrics. These are the numbers that are directly tied to your business model. For a SaaS company, it might be the conversion rate from free trial to paid subscription, the customer lifetime value, or the net revenue churn. For an e-commerce business, it might be the average order value, the repeat purchase rate, or the customer acquisition cost.
These metrics are harder to measure and harder to influence. They require a deep understanding of your customers and your business. But they’re the only numbers that matter. They’re the ones that will tell you if you’re actually building a sustainable business.
Building a Predictive Analytics Engine
So if AI dashboards are the problem, what’s the solution? The solution is to stop looking for a magic box and start building your own predictive analytics engine. This isn’t as intimidating as it sounds. It doesn’t require a team of PhDs in machine learning. It just requires a different mindset.
Instead of starting with the data, you start with the questions. What are the most important questions you need to answer to grow your business? What are the key drivers of success for your company? What are the biggest risks you face?
Once you have your questions, you can start to identify the data you need to answer them. This is where most people go wrong. They try to collect everything, and they end up drowning in a sea of noise. The key is to be ruthless in your focus. Only collect the data that is directly relevant to your key questions.
At MovieLaLa, which was acquired by Gfycat, we wanted to predict which movies would be box office hits. We started by brainstorming all the factors that could influence a movie’s success: the cast, the director, the genre, the marketing budget, the release date, the critical reviews, the social media buzz. Then we built a model that took all of this data and predicted the opening weekend box office revenue.
It wasn’t perfect, but it was a hell of a lot better than guessing. It allowed us to focus our marketing efforts on the movies that had the highest probability of success. It made a huge difference for us.
Building a predictive analytics engine is an iterative process. You start with a simple model, and you gradually make it more sophisticated as you collect more data and learn more about your business. The goal is not to build a perfect crystal ball. The goal is to build a tool that helps you make better decisions.
The Human in the Loop
Here’s the final, and most important, piece of the puzzle: you can’t take the human out of the loop. Data is a powerful tool, but it’s not a substitute for intuition, experience, and common sense. The best analytics systems are the ones that combine the power of machines with the wisdom of humans.
I’ve seen too many founders become slaves to their data. They’re so focused on optimizing their metrics that they lose sight of the bigger picture. They forget that they’re building a product for real people, with real needs and emotions.
Your data can tell you what your users are doing, but it can’t tell you what they’re thinking or feeling. To get that, you have to talk to them. You have to get out of the building and have real conversations with your customers. You have to watch them use your product. You have to understand their hopes, their fears, and their dreams.
I make it a point to talk to at least five customers every week. It’s the most important thing I do. It’s where I get my best ideas. It’s where I find the stories behind the numbers. It’s where I remember that I’m not just building a business, I’m solving a problem for people I care about.
So, by all means, use data. Crush it. Swim in it. But don’t let it drown you. Don’t let it blind you to the human truth at the heart of your business. The goal is not to build a perfect AI dashboard. The goal is to build a business that matters. And for that, you need more than just data. You need courage, you need conviction, and you need a deep and abiding empathy for the people you serve.
Frequently Asked Questions
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.