How to Conduct User Research That Goes Deeper Than "I want AI"

I once watched a startup burn through $10 million in venture capital because they listened to their users too literally. They were building an AI-powered tool for sales teams. In every interview, customers would say, “We need AI to help us close more deals.” So, the founders built what they thought was the ultimate AI sales assistant. It had predictive lead scoring, automated email follow-ups, and a fancy dashboard that showed a bunch of charts and graphs. It looked impressive. The problem? No one used it.

The company eventually folded. I was one of the angel investors, so I had a front-row seat to the whole disaster. It was a painful lesson, but it taught me something that has shaped my entire approach to building products and investing in companies: when it comes to user research, you have to go deeper than the surface-level requests. Especially in the age of AI, where everyone is clamoring for the latest and greatest technology without really understanding what it can do for them.

After that experience, I became obsessed with understanding why so many AI products fail. I spent months analyzing over 1,000 failed AI startups. I read post-mortems, talked to founders, and dug into the data. What I found was a shocking pattern that conventional wisdom completely misses. The data points to one uncomfortable truth about why most AI products never find traction: they’re building solutions for problems that don’t actually exist.

The Allure of the "AI" Buzzword

Look, I get it. AI is exciting. It feels like we’re on the cusp of a new technological revolution. And when you’re a founder, you want to be at the forefront of that change. But here’s the thing: your customers don’t care about AI. They care about their problems. They care about their goals. They care about their own lives and businesses. AI is just a means to an end, and if you’re not crystal clear on what that end is, you’re going to end up building something that nobody wants.

When a user says, “I want AI,” what they’re really saying is, “I have a problem, and I think AI might be the solution.” Your job as a product builder is to figure out what that underlying problem is. It’s not to take their request at face value and start building the first thing that comes to mind. That’s how you end up with a $10 million failure on your hands.

I’ve seen this happen time and time again. A founder will come to me with a pitch for an “AI-powered” everything. An AI-powered to-do list. An AI-powered note-taking app. An AI-powered coffee maker. I’m not even kidding about that last one. And when I ask them what problem they’re solving, they’ll often give me a vague answer about “optimizing workflows” or “boosting productivity.” That’s a huge red flag. It tells me they haven’t done the hard work of understanding their users.

Think of it like this: if you were a doctor and a patient came to you and said, “I need a prescription for painkillers,” you wouldn’t just write the prescription without asking any questions. You’d want to know where the pain is, how long it’s been there, and what might be causing it. You’d run tests. You’d do a thorough diagnosis. The same principle applies to building products. You have to be a detective, not just an order-taker.

Beyond the Surface: A Better Way to Do User Research

So, how do you get beyond the surface-level requests and uncover the real, deep-seated needs of your users? It starts with changing your mindset. Instead of thinking of yourself as a builder, think of yourself as a problem-finder. Your goal is not to build a product; it’s to solve a problem. The product is just a byproduct of that process.

One of a very powerful frameworks I’ve found for this is the “Jobs to Be Done” (JTBD) framework. The basic idea is that customers “hire” products to do “jobs” for them. For example, you don’t buy a drill because you want a drill; you buy a drill because you want a hole in your wall. The hole is the job to be done. The drill is just the tool you hire to do it.

When you start thinking in terms of jobs, it forces you to focus on the user’s underlying motivation. It helps you understand what they’re really trying to accomplish. And once you understand that, you can start to design a solution that truly meets their needs. For more on this, you can check out my post on how I became the #1 angel investor of 2024.

Let’s take the example of the sales team from the beginning of this article. Their stated need was “AI to help us close more deals.” But what was the real job to be done? After talking to a few of the salespeople, I discovered that their biggest challenge was not a lack of leads or a poor sales process. It was a lack of time. They were spending hours every day on administrative tasks like data entry and scheduling meetings. That was the real problem. The job to be done was to free up their time so they could focus on what they do best: selling.

Once you understand the job, you can start to brainstorm solutions. And maybe an AI-powered tool is the right solution. But maybe it’s not. Maybe a simpler, non-AI solution would be more effective. The point is, you don’t know until you’ve done the research.

But the JTBD framework is just the starting point. To really go deep, you need to get your hands dirty. You need to talk to users, observe them in their natural habitat, and become an expert in their world. Here are a few practical tips that I’ve found to be incredibly effective:

• Ask “why” at least five times. This is a classic technique from the Toyota Production System, and it’s just as applicable to product development. When a user tells you they want a certain feature, don’t just take their word for it. Keep asking “why” until you get to the root of the problem. You’ll be amazed at what you uncover. I once had a user tell me they wanted a button to export their data to a CSV file. Instead of just building the button, I asked them why. It turned out they were manually importing the data into another tool to create a specific type of chart. By understanding their ultimate goal, we were able to build the chart directly into our product, saving them a ton of time and effort.

• Look for workarounds and hacks. People are incredibly resourceful. When they have a problem that isn’t being solved by existing tools, they’ll often come up with their own creative solutions. These workarounds are a goldmine of insights. They show you where the real pain points are and what a better solution might look like. At my first startup, RemoteTeam, we noticed that our users were using a combination of spreadsheets, email, and calendar invites to manage their remote employees. It was a clunky, inefficient system, but it showed us that there was a real need for a dedicated tool to solve this problem. That’s how RemoteTeam was born.

• Ignore what they say; watch what they do. People are notoriously bad at articulating their own needs. They’ll often say they want one thing, but their behavior tells a completely different story. That’s why it’s so important to observe users in their natural environment. Watch them work. See where they get stuck. Pay attention to their frustrations. Their actions will tell you more than their words ever could. I’m a big fan of a technique called “shadowing,” where you literally follow a user around for a day and watch them do their job. It’s amazing what you can learn by simply observing.

• Become the user. This is my personal favorite. If you really want to understand your users, you need to walk a mile in their shoes. If you’re building a tool for designers, learn how to design. If you’re building a tool for writers, start a blog. By immersing yourself in their world, you’ll gain a level of empathy and understanding that you just can’t get from interviews and surveys. When we were building MovieLaLa, a social network for movie lovers, I spent months watching hundreds of movies, reading reviews, and talking to other movie fans. I became a true movie buff, and that helped me build a product that resonated with our target audience.

A Tale of Two Startups

I want to leave you with a concrete example of how this plays out in the real world. A few years ago, I was advising two early-stage startups in the same space. Both were building tools to help people manage their personal finances. Both had raised a small seed round. But their approaches to user research couldn’t have been more different.

The first startup, let’s call them “MoneyBot,” was obsessed with AI. They spent months building a sophisticated algorithm that could analyze a user’s spending habits and provide personalized recommendations. They were convinced that this was the future of personal finance. They had a team of brilliant data scientists and machine learning engineers. They even had a PhD in behavioral economics on their advisory board. On paper, they looked unstoppable.

But when they launched, they were met with a resounding “meh.” People just weren’t that interested. The feedback was lukewarm. “It’s kind of cool, I guess,” was a common refrain. But nobody was using it consistently. The team was baffled. They had built what they thought was the perfect product. What went wrong?

The second startup, “Penny,” took a completely different approach. Instead of starting with the technology, they started with the user. They spent weeks interviewing people about their financial struggles. They didn’t just talk to them in a sterile office environment. They went to their homes. They sat with them at their kitchen tables. They looked at their bills. They listened to their stories.

They learned that one of the biggest challenges people faced was simply keeping track of their subscriptions. They were signing up for free trials and then forgetting to cancel, resulting in hundreds of dollars in unwanted charges every year. It was a small problem in the grand scheme of things, but it was a real, painful problem. And it was a problem that nobody was solving well.

So, Penny built a simple tool that did one thing and one thing only: it helped people find and cancel their unwanted subscriptions. It wasn’t as sexy as MoneyBot’s AI-powered assistant, but it solved a real, painful problem. And because of that, it took off like a rocket. Today, Penny is a thriving business with millions of users. MoneyBot, on the other hand, is just another failed startup in the AI graveyard.

The Human Element: Why AI Can't Replace True Empathy

I'm not an anti-AI guy. Far from it. I've invested in some of the biggest names in the space, including Anthropic and OpenAI. I believe that AI has the potential to solve some of the world's most pressing problems. But I also believe that it's not a silver bullet. It's a tool, and like any tool, it's only as good as the person wielding it.

At the end of the day, building a successful product is not about having the best technology. It's about having the deepest understanding of your users. It's about empathy. It's about connecting with people on a human level and truly understanding their hopes, their fears, and their dreams. That's something that no algorithm can do. At least not yet.

So, by all means, explore the possibilities of AI. But don't let it be a substitute for genuine human connection. Get out of the building. Talk to your users. Listen to their stories. And for the love of God, don’t just build another AI-powered to-do list.

The Takeaway

The moral of the story is this: don’t be like MoneyBot. Don’t fall in love with the technology. Fall in love with the problem. If you can do that, you’ll be well on your way to building a product that people actually want to use. And who knows, you might even build the next Penny. For more on my investment philosophy, check out my post on my investment in Anthropic.

Building a great product is hard. It takes time, effort, and a relentless focus on the user. There are no shortcuts. But if you’re willing to do the hard work of understanding your users on a deep, human level, you’ll be well on your way to building something truly special. And that, my friends, is a feeling that no amount of money can buy.

Frequently Asked Questions

What tools do I need to get started?

Start with the basics. You don't need expensive software or fancy tools. A spreadsheet, a note-taking app, and direct access to your customers will get you further than any enterprise platform. Add tools only when you hit a specific bottleneck.

How do I measure success with this approach?

Pick one or two metrics that directly tie to your goal and track them weekly. Vanity metrics like page views or follower counts rarely matter. Focus on metrics that reflect real engagement or revenue impact.

How long does it take to conduct user research that goes deeper than 'i want ai'?

The timeline varies depending on your starting point and resources. For most founders, expect 2-4 weeks for initial setup and 2-3 months to see meaningful results. I've seen teams move faster when they focus on one thing at a time rather than trying to do everything at once.