Here's something nobody tells you about how to build multimodal llms that don't suck: the conventional wisdom is mostly backwards.
Building multimodal LLMs is the next frontier, but most attempts are clunky and impractical. I'm sharing my playbook for creating seamless, intuitive multimodal experiences that users will actually love, based on my experience shipping three of them.
The Reality Nobody Talks About
Most people approach how to build multimodal llms that don't suck with assumptions that made sense five years ago. The world has moved on. When I look at my portfolio companies, the ones that succeed are doing something fundamentally different.
The first thing to understand is that customer feedback is the only metric that matters. I've seen this play out across dozens of companies. The pattern is unmistakable.
At RemoteTeam, we learned this the hard way. We spent months going down the wrong path before realizing that the market doesn't care about your roadmap. Once we made the switch, everything changed.
The Counterintuitive Truth
Here's what surprised me most about how to build multimodal llms that don't suck: the best practitioners do less, not more.
When I was building MovieLaLa, we tried to do everything at once. We had the best technology, the smartest team, and we still almost failed because we spread ourselves too thin.
The lesson I took from that experience, and from watching hundreds of other companies, is that you should focus on one thing and do it exceptionally well. It sounds simple. It's incredibly hard to execute.
What I Tell Founders
When a founder in my portfolio asks me about how to build multimodal llms that don't suck, I usually start with three questions:
- What's your timeline? Because the right approach for a company with 6 months of runway is very different from one with 3 years.
- What have you already tried? Most founders have tried something. Understanding what didn't work is often more valuable than knowing what might.
- Who on your team owns this? If the answer is "everyone" or "no one," that's your first problem to solve.
These questions seem simple but they reveal a lot about where a company actually stands.
This connects to broader themes around small language models, multimodal LLMs, on-device AI that I've been thinking about a lot lately.
Wrapping Up
I've shared a lot here, and I know it can feel overwhelming. But here's the thing about how to build multimodal llms that don't suck: you don't need to get everything right on day one. You just need to get started and keep improving.
The founders in my portfolio who excel at how to build multimodal llms that don't suck share one trait: they're relentlessly practical. They don't chase perfection. They chase progress.
That's the mindset I'd encourage you to adopt. Start where you are. Use what you have. Do what you can. And keep pushing forward.
As always, I'm rooting for you.
Frequently Asked Questions
How long does it take to build multimodal llms that don't suck (a practical guide) #16?
The timeline varies depending on your starting point and resources. For most founders, expect 2-4 weeks for initial setup and 2-3 months to see meaningful results. I've seen teams move faster when they focus on one thing at a time rather than trying to do everything at once.
Do I need technical skills to build multimodal llms that don't suck (a practical guide) #16?
Not necessarily. While technical understanding helps, the most important skills are clear thinking and the ability to break problems into smaller pieces. Many successful founders I've invested in started with zero technical background and either learned enough to be dangerous or found the right technical partner.
How do I measure success with this approach?
Pick one or two metrics that directly tie to your goal and track them weekly. Vanity metrics like page views or follower counts rarely matter. Focus on metrics that reflect real engagement or revenue impact.
What tools do I need to get started?
Start with the basics. You don't need expensive software or fancy tools. A spreadsheet, a note-taking app, and direct access to your customers will get you further than any enterprise platform. Add tools only when you hit a specific bottleneck.