I’m going to tell you something that most VCs and AI vendors won’t. I wasted $200,000 on AI data analytics. Straight up. Flushed it down the toilet chasing shiny dashboards and promises of god-like prediction.
It was 2022. We were building RemoteTeam, and like every other startup founder in Silicon Valley, I was convinced AI was the magic bullet. We hired a team of data scientists, bought subscriptions to three different analytics platforms, and built the most beautiful dashboards you’ve ever seen. They had real-time graphs, predictive forecasts, and more dials than a 747 cockpit. And they were completely useless.
For 18 months, I stared at those screens, watching metrics go up and to the right, but I couldn’t make a single decision with confidence. It was all noise. We were data-rich and insight-poor. After burning through enough cash to fund a seed round, I pulled the plug on the whole operation. I had to face the hard truth: I had been sold a lie. And I’m willing to bet you are too.
But that failure taught me more than any success ever could. I rolled up my sleeves and went back to first principles. I talked to founders who had actually built data-driven businesses. I spent nights learning SQL and Python. And I discovered five brutal truths about AI data analytics that nobody talks about. These truths turned my messy data into rocket fuel for growth and were instrumental in our acquisition by Gusto. Here they are.
1. Dashboards Are Glorified Noise Machines
Your dashboard is a vanity project. There, I said it. It looks impressive to your board and makes you feel like you’re in control, but it’s mostly theater. Most dashboards are just a collection of lagging indicators—vanity metrics like page views, sign-ups, and daily active users. They tell you what happened, but they almost never tell you why.
I remember a board meeting where one of our investors pointed to a chart showing a 30% month-over-month increase in user engagement. He was ecstatic. But I had to be the one to burst his bubble. The “engagement” was coming from a new feature that was so confusing, users were clicking around in frustration trying to figure it out. The metric was up, but the user experience was in the toilet. The dashboard didn’t show that.
The fix: Stop obsessing over dashboards and start focusing on diagnostics. For every metric, you need to be able to answer the question, "So what?". Instead of a chart showing user growth, build a cohort analysis that shows you retention by acquisition channel. Instead of a big number showing revenue, create a breakdown of revenue by customer segment. The goal isn’t to admire the data; it’s to argue with it. Your data should be a tool for debate, not a source of confirmation bias.
2. "Predictive" Is a Marketing Buzzword
Every AI analytics tool on the market claims to have "predictive" capabilities. They promise to tell you which customers will churn, which leads will convert, and what your revenue will be next quarter. It’s a seductive pitch. It’s also mostly nonsense.
Most of what’s sold as "predictive analytics" is just slightly more sophisticated trend-following. The models look at historical data and extrapolate it into the future. That works fine in a stable environment, but we don’t live in a stable environment. The market changes, your product changes, your customers change. Any model built on last year’s data is going to be wrong about next year.
We spent $50,000 on a "predictive" churn model that had a 90% accuracy rate in backtesting. We were thrilled. Then we deployed it. In the first month, it was right about 50% of the time—no better than a coin flip. Why? Because we had just released a major product update that fundamentally changed user behavior. The model was a dinosaur, perfectly adapted to a world that no longer existed.
The fix: Treat every prediction with extreme skepticism. True prediction is about understanding causal relationships, not just correlations. Instead of asking "what will happen?", ask "what will happen if?". For example, instead of predicting churn, build a model that answers the question: "What will happen to churn if we offer a 20% discount to at-risk customers?". This forces you to think about the levers you can pull to change the future, not just passively observe it.
3. Your Data Is a Hot Mess
You think you have good data? You don’t. I don’t care if you’re Google, your data is a mess. It’s incomplete, it’s inconsistent, and it’s full of errors. The fancy AI algorithms that vendors love to talk about are useless if you’re feeding them garbage.
At MovieLaLa, we wanted to build a recommendation engine. We had a database of millions of movie ratings from our users. We thought it would be easy. Then we looked at the data. We had users rating the same movie multiple times with different scores. We had movies with titles in three different languages. We had ratings that were timestamps. It was a disaster. We spent six months—and a good chunk of our seed funding—just cleaning the data before we could even think about building a model. This is the unglamorous, 90% of the work that nobody wants to talk about.
The fix: Embrace the mess. Data cleaning isn’t a one-time project; it’s a continuous process. You need to invest in data infrastructure and tooling that makes it easy to find and fix errors. You need to create a culture of data ownership, where everyone in the company is responsible for the quality of the data they produce. And you need to be realistic about the limitations of your data. Sometimes, a simple heuristic based on messy data is better than a complex model that assumes the data is perfect.
4. Simple Models Are Your Best Friend
Data scientists love to build complex models. They’ll talk your ear off about neural networks, gradient boosting, and support vector machines. It makes them feel smart, and it justifies their high salaries. But in my experience, the simplest model is almost always the best one.
Why? Two reasons. First, simple models are easier to understand and explain. If your model is a black box, you can’t trust its predictions. You don’t know why it’s making the decisions it’s making, and you can’t debug it when it goes wrong. A simple linear regression or a decision tree, on the other hand, is transparent. You can look at the coefficients or the branches and understand exactly how it works.
Second, simple models are less likely to overfit. Overfitting is the cardinal sin of machine learning. It’s when your model learns the noise in your data, not the signal. It performs great on your training data, but it fails miserably in the real world. Complex models are overfitting machines. They have so many parameters that they can memorize your entire dataset. Simple models, by their very nature, are forced to find the real underlying patterns.
The fix: Always start with the simplest model possible. A good rule of thumb is to use a model that you could explain to your CEO on a whiteboard. Only add complexity if you have a very good reason to believe it will lead to a significant improvement in performance. And even then, be skeptical. More often than not, the extra complexity isn’t worth the cost.
5. It’s About Questions, Not Tools
The final and most important truth is this: the tool is the last thing you should be thinking about. The first thing you should be thinking about is the question. What is the one decision you need to make that will have the biggest impact on your business? Start there.
For years, I was obsessed with tools. I was always looking for the next shiny object, the next platform that would solve all my problems. I was a tool-chaser, not a problem-solver. It was only when I stopped asking "what tool should I use?" and started asking "what question do I need to answer?" that I started to make progress.
At RemoteTeam, our biggest challenge was sales. We were getting a lot of leads, but our conversion rate was terrible. We could have thrown a bunch of AI at the problem. We could have built a lead scoring model, a predictive dialer, a sentiment analysis tool. Instead, we asked a simple question: "What is the one thing that our best customers have in common?".
We spent a week manually going through our CRM data. We looked at company size, industry, location, job titles. And we found it. The one thing that all our best customers had in common was that they had all recently hired a remote head of HR. That one insight was worth more than all the AI dashboards in the world. We changed our marketing to target companies that were hiring for that role, and our conversion rate tripled overnight.
The fix: Fall in love with your problem, not your solution. Before you write a single line of code or sign a contract with a vendor, write down the question you are trying to answer. Make it as specific as possible. Then, and only then, start thinking about the data, the model, and the tool you need to answer it.
So there you have it. Five brutal truths that I learned after burning $200,000. It was an expensive education, but it was worth it. Today, I don’t have any fancy dashboards. I don’t have a team of data scientists. But I have a deep understanding of my business, and I can make decisions with confidence. And that’s something you can’t put a price on.
Frequently Asked Questions
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.