I’ve seen it a hundred times. A founder gets a brilliant idea for an AI product. They spend six months and a million dollars in venture funding building a sophisticated model, only to find out that nobody wants to use it. The AI landscape is littered with the ghosts of these well-intentioned, but ultimately misguided, projects.
What if I told you there’s a way to validate your AI product idea in a week, with virtually no code? It’s called the “concierge test,” and it’s one of the most powerful, yet underutilized, tools in the lean startup playbook. I’ve used it to launch my own successful companies, and I’ve seen it work for countless others in my 200+ angel investments.
What is a Concierge Test?
The concierge test is a form of manual-first user research. Instead of building a fully automated AI product, you manually perform the service for your first few customers. You act as the “concierge,” delivering the value proposition of your product by hand. This could mean manually analyzing data, writing reports, or providing recommendations—whatever it is your AI is supposed to do.
It might sound like a lot of work, and it is. But it’s a fraction of the work of building a product that nobody wants. The goal of the concierge test is not to scale, but to learn. You get to interact directly with your customers, understand their pain points, and see if your solution actually solves a real problem for them. You get to see what they’re willing to pay for, and what they’re not.
My First Concierge Test: RemoteTeam
When we were starting RemoteTeam, which was later acquired by Gusto, we had a big vision for an all-in-one platform for managing remote teams. But we didn’t start by building the whole thing. We started with a simple concierge test.
We found a few companies that were struggling with managing their remote employees. We told them we had a new platform that could help. But in reality, the “platform” was just me and a spreadsheet. I would manually onboard their employees, track their time, and process their payroll. It was a ton of manual work, but it was invaluable.
We learned that our initial assumptions were all wrong. We thought the biggest pain point was time tracking, but it turned out to be international payments. We also learned that our customers were willing to pay a lot more than we expected for a solution that solved this problem. This early feedback was instrumental in shaping the product and led to our successful acquisition.
How to Run a Concierge Test for Your AI Product
So, how do you run a concierge test for your AI product? Here’s a step-by-step guide:
Define your value proposition. What is the core problem you’re solving for your customers? What is the unique value that your AI will provide? Be as specific as possible.
Find your first customers. Don’t try to boil the ocean. Find a small group of early adopters who are feeling the pain of the problem you’re trying to solve. You can find them in online communities, at industry events, or through your personal network.
Deliver the service manually. This is the core of the concierge test. You are the AI. You do the work. Don’t worry about making it pretty. Just focus on delivering the value proposition.
Get feedback. This is the most important part. Talk to your customers. What did they like? What did they not like? Would they pay for this service? How much? Their feedback is gold.
Iterate. Based on the feedback, refine your value proposition and your service. You might need to run several iterations of the concierge test before you’re ready to build a product.
A Word of Warning
The concierge test is not a silver bullet. It’s a tool, and like any tool, it can be misused. Here are a few things to watch out for:
- Don’t get stuck in the concierge phase. The goal is to learn, not to run a manual service forever. Once you’ve validated your idea, you need to start building the product.
- Don’t be afraid to charge. Even a small amount of money can be a powerful validator. If people aren’t willing to pay for your service, they’re probably not going to pay for your product.
- Don’t ignore the feedback. It’s easy to fall in love with your own ideas. But you need to be willing to listen to your customers and pivot if necessary.
The Future of AI is Manual-First
In the rapidly evolving world of AI, the concierge test is more important than ever. It’s a way to cut through the hype and focus on what really matters: building products that people want and need. It’s a way to de-risk your startup and increase your chances of success.
So before you go and raise a bunch of money to build your next big AI idea, take a step back. Run a concierge test. You might be surprised by what you learn. It might just be the thing that saves your startup from the AI graveyard.
Frequently Asked Questions
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.