The Fast demo model is already very impressive. It was way better than expected, but still required being a bit verbose since it didn't seem to understand rarer words ("sauna" didn't get me anything pleasant, "hot sauna" did).
The generated palette seem to be a great indicator of whether the model understood the prompt or not.
I Haven't checked out the Python SDK yet, but it seems very interesting!
I'm curious to know if there is any reason for why you picked Gemma 1B for the Expressive model. Did it generate more cohesive parameters than other 1B models? Or was it just the first one you picked?
BTW - we used Gemma 270m model - not the 1B model. It's sheerly the size - I wanted to see if I could get a really, really tiny LLM to generate coherent music. Tbh, it didn't quite work as well as I expected. It barely beats a randomly generated track.
In fact, the 'fast' model (literally embedding lookup over a pre-generated library of music ... generated using Best-of-N on Gemini Flash) beats nearly everything - including Gemini Flash, Claude Opus, Gemma models.
Yeah! That's one of the reasons I've exposed the Python SDK ... you could, in theory, even attach some sort of DAW that manipulates the music in real-time.
Thats pretty cool!
Some games like Gothic had dynamic composing of existing sound themes.
It would be interesting to research how these could fit together
It's just that the 'Fast' model picks up from a library of existing tracks using a small embedding model.
If you're really curious, you could try the 'Custom LLM' feature - specifically 'Gemini Flash'. Here's the track it created when I gave it the same input.
It titled it, "In the mood for country cowboy-ish music played for someone like John Wick bleeding out on a cold, snow-covered park bench." and here's the justification it came up with, "A weary, grit-laden Dorian lament for a fallen gunslinger, blending the cold isolation of vast reverb with the sharp, intimate pluck of a final, fading heartbeat."
Yeah ... that's a bit of an issue - it's because I built a custom simple synthesizer that's not quite as good as a full blown one. Thank you for the feedback! This is something I intend to tackle in upcoming releases.
I really like the idea. But my one attempt was disappointing. "playful energetic urban fantasy at night" ended up set to "very slow" by default.
I would really like to be able to run this on my phone. Use my Brilliant smart glasses to periodically take a picture, ask a model to describe the mood/setting, and get an ambient stream to match the mood.
The 'Fast'/embedding model, unless the input is absolutely clear, snaps to the closest alternative.
Here's the song it created when I asked Gemini Flash to do it!
It called it, "Cobalt Kinetic" and explained that, "A high-velocity Mixolydian engine fueled by syncopated electronic pulses and glitchy accents, evoking the iridescent blur of hidden magic weaving through a midnight skyscraper canyon."
Really elegant approach - mapping sentence embeddings to a deterministic synth feels more like building an instrument than generating content, and the instant playback makes it great for flow.
Would love to know if the same prompt always yields the same sound (reproducibility could be powerful), and whether you’ve considered semantic morphing between two moods over time.
It's just that the 'Fast' model picks up from a library of existing tracks using a small embedding model.
If you're really curious, you could try the 'Custom LLM' feature - specifically 'Gemini Flash'. Here's the track it created when I gave it the same input.
It titled it, "Throne of the Eternal Night" and here's the justification it came up with, "A glacial, minor-key procession of heavy brass hits and deep sub-pulses, evoking the suffocating grandeur of an ancient, blood-soaked dynasty ending in shadow."
Why does no one byok for mistral? Have they kinda "lost" the race already?
But looks great! I'm keen to actually give it a go.
I'm incredibly against AI music for various reasons, but some ongoing ambient stuff...I'll at least try it out!
The Fast demo model is already very impressive. It was way better than expected, but still required being a bit verbose since it didn't seem to understand rarer words ("sauna" didn't get me anything pleasant, "hot sauna" did).
The generated palette seem to be a great indicator of whether the model understood the prompt or not.
I Haven't checked out the Python SDK yet, but it seems very interesting!
I'm curious to know if there is any reason for why you picked Gemma 1B for the Expressive model. Did it generate more cohesive parameters than other 1B models? Or was it just the first one you picked?
BTW - we used Gemma 270m model - not the 1B model. It's sheerly the size - I wanted to see if I could get a really, really tiny LLM to generate coherent music. Tbh, it didn't quite work as well as I expected. It barely beats a randomly generated track.
In fact, the 'fast' model (literally embedding lookup over a pre-generated library of music ... generated using Best-of-N on Gemini Flash) beats nearly everything - including Gemini Flash, Claude Opus, Gemma models.
I kinda liked how well you can fine-tune parameters of the music. Could be useful as dynamic soundtracks for games in low resource settings
Yeah! That's one of the reasons I've exposed the Python SDK ... you could, in theory, even attach some sort of DAW that manipulates the music in real-time.
Thats pretty cool! Some games like Gothic had dynamic composing of existing sound themes. It would be interesting to research how these could fit together
I tried, “In the mood for country cowboy-ish music played for someone like John Wick bleeding out on a cold, snow-covered park bench.”
I ended up with kinda shrill. I was hoping for something that would sound like I’m listening to something while the coffee gets cold in a cabin.
It's just that the 'Fast' model picks up from a library of existing tracks using a small embedding model.
If you're really curious, you could try the 'Custom LLM' feature - specifically 'Gemini Flash'. Here's the track it created when I gave it the same input.
It titled it, "In the mood for country cowboy-ish music played for someone like John Wick bleeding out on a cold, snow-covered park bench." and here's the justification it came up with, "A weary, grit-laden Dorian lament for a fallen gunslinger, blending the cold isolation of vast reverb with the sharp, intimate pluck of a final, fading heartbeat."
Music link: https://vocaroo.com/1nepiOQ3mrCr
I get clicks and pops every few seconds, using Librewolf.
But otherwise very cool!
Yeah ... that's a bit of an issue - it's because I built a custom simple synthesizer that's not quite as good as a full blown one. Thank you for the feedback! This is something I intend to tackle in upcoming releases.
Same, on Vanadium. Very cool nonetheless
Thank you!!!
I really like the idea. But my one attempt was disappointing. "playful energetic urban fantasy at night" ended up set to "very slow" by default.
I would really like to be able to run this on my phone. Use my Brilliant smart glasses to periodically take a picture, ask a model to describe the mood/setting, and get an ambient stream to match the mood.
The 'Fast'/embedding model, unless the input is absolutely clear, snaps to the closest alternative.
Here's the song it created when I asked Gemini Flash to do it!
It called it, "Cobalt Kinetic" and explained that, "A high-velocity Mixolydian engine fueled by syncopated electronic pulses and glitchy accents, evoking the iridescent blur of hidden magic weaving through a midnight skyscraper canyon."
Here's the audio: https://vocaroo.com/1hjtPAxLWqyA
Really elegant approach - mapping sentence embeddings to a deterministic synth feels more like building an instrument than generating content, and the instant playback makes it great for flow.
Would love to know if the same prompt always yields the same sound (reproducibility could be powerful), and whether you’ve considered semantic morphing between two moods over time.
Thanks!
The same prompt yields largely the same song because the 'Fast' (default) mode retrieves the synth parameters from a pre-existing library.
But if you use the 'Custom LLM' model, it can generate new and creative music every time you play something - even for the same input!
I like the concept, but it's not picking up what I'm putting down...
'"epic vampiric doom" suggests a bright and uplifting soundscape'? I'm not so sure about that
... I was hoping for something more like nightwish.
It's just that the 'Fast' model picks up from a library of existing tracks using a small embedding model.
If you're really curious, you could try the 'Custom LLM' feature - specifically 'Gemini Flash'. Here's the track it created when I gave it the same input.
It titled it, "Throne of the Eternal Night" and here's the justification it came up with, "A glacial, minor-key procession of heavy brass hits and deep sub-pulses, evoking the suffocating grandeur of an ancient, blood-soaked dynasty ending in shadow."
Music link: https://vocaroo.com/14bW1RJyubcC
Server is down :(
Could you try again, please? Just redeployed with a tiny change. Should work now!