I once built a four-figure side business on a generative music technique that most of the so-called “gurus” swore was a dead end. They called it impossible, a waste of time. I just quietly nodded, went back to my desk, and kept cashing the checks. Ready to have your mind blown? This is the story of how I did it, and how you can do it too.
Let's get real about generative music composition. It's not about having the fanciest tools or a multi-million dollar budget. I’ve seen more talent and innovation come out of a dorm room with a beat-up laptop than from a corporate R&D lab with a blank check. The truth is, the barrier to entry is lower than ever, but the barrier to quality is as high as it's always been. That’s what this cheat sheet is all about: the fundamental principles that took me from a curious tinkerer to a recognized expert in the field.
The Great Lie the ‘Experts’ Are Selling You
Before we get into the nitty-gritty, we need to clear the air. There’s a lot of noise out there, a lot of people trying to sell you on complex, convoluted systems that are more about making them look smart than about helping you create something beautiful. They’ll tell you that you need a deep understanding of music theory, a PhD in machine learning, and a supercomputer in your basement. It’s all nonsense.
I’ve been in the tech world for a long time. I’ve seen companies rise and fall, and I’ve invested in over 200 of them, including some of the biggest names in AI like Anthropic, OpenAI, and Scale AI. And I can tell you this: the most successful people, the ones who really change the game, are the ones who can cut through the complexity and find the simple, elegant solution. The same is true for generative music.
My Generative Music Cheat Sheet
So, here it is. My personal cheat sheet for generative music composition. These are the principles I’ve used to build my own projects, and the same principles I look for when I’m considering an investment in a new AI music startup.
1. It’s All About the Data
This is the big one. The single most important factor in the quality of your generative music is the quality of your data. You can have the most sophisticated algorithm in the world, but if you feed it garbage, it’s going to spit out garbage. It’s the classic GIGO principle: garbage in, garbage out.
I learned this lesson the hard way. In the early days of MovieLaLa, we were trying to build a system that could generate custom soundtracks for movie trailers. We scraped a massive dataset of movie scores, fed it into our model, and the results were… well, let’s just say they weren’t going to win any Oscars. It was a chaotic mess of noise.
We spent months trying to tweak the algorithm, but nothing worked. Finally, we went back to the drawing board and took a hard look at our data. We realized that we had a hodgepodge of different styles, genres, and recording qualities. We were trying to teach our model to be a jack-of-all-trades, and it was mastering none of them.
So, we started over. We curated a small, high-quality dataset of scores from a single genre: epic action-adventure. We made sure every track was professionally recorded and mastered. And the results were night and day. The music was coherent, emotionally resonant, and perfectly suited for the trailers we were creating.
- Actionable takeaway: Don’t just grab the biggest dataset you can find. Curate your data. Be ruthless. Your model will thank you for it.
2. The Human Element is Your Secret Weapon
There’s a fear among some musicians that AI is going to make them obsolete. I see it differently. I see AI as a collaborator, a tool that can augment and enhance human creativity, not replace it. The best generative music systems are the ones that keep the human in the loop.
Think of it like this: an AI can generate a thousand different melodies in the time it takes a human to write one. But a human can listen to those thousand melodies and identify the one that has real heart, the one that tells a story. That’s a skill that no machine can replicate.
One of my most successful angel investments was in a company that built a generative music platform for video game developers. Their secret sauce wasn’t their AI, it was their user interface. They made it incredibly easy for developers to guide the AI, to give it feedback, and to shape the music to fit their creative vision. They weren’t just generating music, they were creating a conversation between the human and the machine.
- Actionable takeaway: Don’t try to automate everything. Embrace the human element. Build systems that allow for collaboration and feedback.
3. Constraints are Your Friend
When you’re faced with a blank canvas, the sheer number of possibilities can be paralyzing. The same is true for generative music. If you just let your model run wild, you’re going to end up with a lot of random notes. The magic happens when you introduce constraints.
Think about the structure of a pop song. You have a verse, a chorus, a bridge. You have a specific key, a specific tempo. These constraints don’t limit creativity, they channel it. They provide a framework that allows for endless variation within a familiar structure.
When I was building my four-figure side business, the “impossible” technique was all about constraints. I was creating custom-length tracks for podcasters and YouTubers. They needed music that could loop seamlessly for a specific duration, without sounding repetitive. The gurus said it couldn’t be done. They said you couldn’t generate high-quality music with that level of control.
My solution was simple. I didn’t try to generate the entire track in one go. I generated a library of short, looping musical phrases, all in the same key and tempo. Then, I built a simple tool that allowed me to arrange these phrases into a longer composition, like building with LEGO bricks. The AI did the heavy lifting of creating the raw material, and I provided the creative direction.
- Actionable takeaway: Don’t be afraid of constraints. Use them to your advantage. Break down complex problems into smaller, more manageable pieces.
The Future is a Duet, Not a Solo
So, what does the future of generative music look like? I don’t have a crystal ball, but I can tell you what I’m excited about. I’m excited about tools that empower artists, not replace them. I’m excited about systems that are more like instruments than assembly lines. And I’m excited about a future where anyone, regardless of their technical background, can express themselves through the power of music.
The next big thing in generative music isn’t going to be a new algorithm or a new dataset. It’s going to be a new way of thinking. It’s going to be a shift from a mindset of automation to a mindset of collaboration. The future of music isn’t a solo performance by a machine. It’s a duet between human and AI.
And that’s a future I’m willing to bet on. In fact, I already have. Many times over.
My Toolkit: The Underdog Stack
People always ask me what tools I use. They expect me to list off a bunch of expensive, enterprise-level software. They're always surprised when I tell them my stack is actually pretty simple. I don't believe in throwing money at problems. I believe in finding the right tool for the job, and often, the simplest tool is the best.
Here are a few of the tools that I’ve found to be invaluable in my generative music projects:
Magenta Studio: This is a great starting point for anyone new to generative music. It’s a collection of plugins for Ableton Live that allow you to experiment with AI-powered music generation in a hands-on way. It’s not the most powerful tool out there, but it’s incredibly intuitive and a lot of fun to play with. It’s how I got my start, and I still go back to it from time to time to spark new ideas.
Jukebox & MuseNet (by OpenAI): These are more advanced models, but they are capable of generating some truly stunning music. Jukebox, in particular, is incredible at generating music with vocals. It’s not perfect, but it’s a glimpse into the future of what’s possible. I’ve used these models to create everything from background music for my YouTube videos to full-fledged compositions. The key is to be patient and willing to experiment. You’re not always going to get a masterpiece on the first try.
AIVA: This is a great example of a company that is building a generative music platform with the human in the loop. AIVA allows you to create music in a variety of styles, and it gives you a lot of control over the final output. You can specify the instrumentation, the tempo, the key, and even the emotional arc of the piece. It’s a powerful tool for composers who want to use AI as a creative partner.
Your Own Brain: This is the most important tool in your arsenal. Don’t ever forget that. AI is a powerful tool, but it’s just that: a tool. It’s up to you to provide the creative vision, the taste, and the storytelling. The best generative music is a collaboration between human and machine. It’s a conversation. And you’re the one leading the conversation.
A Call to Action: Your Turn to Create
I’ve shared my story, my cheat sheet, and my toolkit. Now it’s your turn. The world of generative music is wide open. There are so many opportunities for innovation, so many new sounds to be discovered. Don’t let anyone tell you that you can’t do it. Don’t let the “gurus” intimidate you with their fancy jargon and their expensive tools.
Start small. Pick a project that you’re passionate about. Maybe you want to create a custom soundtrack for your podcast. Maybe you want to generate an endless stream of ambient music for studying. Maybe you just want to see what’s possible. Whatever it is, just start. The only way to learn is by doing.
And when you create something amazing, I want to hear about it. Find me on Twitter. Send me a link. I’m always on the lookout for the next big thing in generative music. Who knows, maybe I’ll even invest in your company someday.
The future is not written. It’s composed. And you’re the composer. Now go make some music.
Frequently Asked Questions
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
How can I apply this thinking to my own situation?
Start by identifying the core principle behind the opinion, not the specific example. Then ask yourself: does this principle apply to my context? If yes, test it in a small, low-risk way before going all in.
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.