Riffusion is an AI-powered text-to-song generator, best compared to Suno and Udio. Like those apps, Riffusion takes in lyrics and descriptive music prompts to generate songs with AI vocalists.
The company launched their initial app version in December 2022, followed by a $4M seed round. They published a V2 web app in October 2023, scrapped it in early 2024 after being eclipsed by Suno's superior product.
In July 2024, they released a mobile app that centered around an image-to-song generation experience. That mobile app has since disappeared from the app store.
A brand new web app was launched in January 2025 and is currently free with unlimited use. We'll explore the complete history of Riffusion over the course of this article, starting with the latest version and moving back to former versions.
Table of Contents
Riffusion launches AI music web app in 2025
The latest version of Riffusion meets all of the criteria that users have come to expect from premium AI song apps like Suno and Udio. You can find a complete overview of the web app in the video above.
During the current "public beta" phase, the company has made their service available for free and without any limitations on use. In fact, they have introduced gamification that rewards super users with bonus personalization features.
Here's a shortlist of core features that Riffusion has implemented, matching the features at Suno and Udio:
Text-to-Music generation
Lyrics-to-song in any style
Audio conditioning (upload your own clip)
AI music extension (continue an existing track)
Section replacement (audio inpainting mid-clip)
Longform tracks (3+ minutes each)
User-curated playlists
Diverse range of genres supported
Social user profiles with likes and followers
Featured tracks from the community
The overall audio and composition quality of Riffusion still falls short of its competitors, but proves to be a major improvement on the web app they had deployed back in October 2023.
Riffusion's AI covers arrange music around a sample
The biggest differentiator we've found is in their AI cover feature. To access it, you'll need to first upload a track and click the dropdown menu next to the waveform view. See the screenshot below for details:
We experimented with this cover feature at length and discovered a few noteworthy details about how it works.
First, the style of music that you target in your prompt should be in the same ballpark as the audio file that you uploaded. Otherwise, the music output will reflect your style prompt and ignore the audio file that you provided.
When uploading a stem, like a guitar line or lead melody, we found that important details like key, tempo and rhythmic articulation were modified. Even if the input melody was retained, Riffusion introduces unwanted artifacts and degrades the signal slightly.
On a positive note, audio conditions with cover mode can unlock behaviors that are otherwise difficult to achieve with prompts alone. When we asked for "odd time signatures", the language model understood but the audio output remained in 4/4. However, if we uploaded an audio condition in an odd meter, Riffusion understood the assignment and generated music in that target time signature.
Riffusion remains silent on licensing & training data
Generative AI music companies have been under intense scrutiny since 2024, after the RIAA sued Suno and Udio for copyright infringement. Both companies admitted to training their models on music that was scraped from the internet without licensing or permission from the rights holders.
There is substantial demand for ethically trained AI music models and if Riffusion had done so, they would likely advertise it as a primary selling point. This is clear from the behavior of other fairly trained companies like Jen Music AI, whose press release centered primarily on this fact alone.
When I asked Riffusion's CTO Hayk Martiros publicly about the model training (in response to a post he made announcing the new version of the app) he opted not to respond. Several other users chimed in and asked for clarification, upvoting the question. One AI music expert, Daniel Rowling of LANDR, opined that Riffusion was not trained on licensed music.
Mobile app released July 2024 and then removed
Riffusion released their first ever AI text-to-song mobile app in July 2024. This experience centered around a photo-to-song capability that anyone could enjoy.
Here's a quick rundown on how the mobile app used to work:
Users snap a picture with their smart phone camera
An AI-powered model writes a caption to describe the image
Riffusion writes lyrics about what it "sees"
Those lyrics are rapped or sung by an AI vocalist
Riffusion creates matching background music
This experience bore resemblance to other AI image-to-music generators like SoundGen and Mubert. Here's a screenshot of the featured page from my own account:
Features in Riffusion Version 2.0
The remainder of this page is a reflection on early version of the Riffusion web app. Some of the V2 features have been cut, like the ability to download stems.
In its previous incarnation, Riffusion asked you to sign up for a free account and then presented you with this dashboard first:
Trending riffs on Riffusion's Explore page were a collection of AI music generated by other users. Hovering over a track, you could read the lyrics like karaoke and hear them performed by an AI voice.
Explore was similar to the mobile app's Featured page today.
Users could click the heart icon to improve that song's rank in the community. Clicking through the creator's profile name, you could see a library with other riffs they had made.
We time stamped a walkthrough of Riffusion below, so it takes you straight to the Explore community page. Have a listen to what the music sounds like and see how the app works.
How did you generate an AI song in Riffusion v2?
To create an AI song in Riffusion v2, you clicked the "plus" icon located at the top left corner of your dashboard. You were encouraged to add lyrics and describe the sound of music you wanted, based on features like genre, vocal style, or overall vibe.
Users would type in lyrics or use a microphone to speak words directly into the app. If you didn't have any ideas about where to start, a one or two word concept was fine. The app included an AI powered "write lyrics" button that created lyrics based on short ideas.
There was a second prompt field called Describe the sound. It had a prompt genius button to help write descriptions of music in a similar way.
Once lyrics were ready, users could hit Riff to generate some music. Within a minute, they'd have three tracks to choose from.
Clicking the play button on any card, they could hear the song played back with AI vocals and instrumentals.
Hovering over a card revealed a few options represented by icons.
The remix feature refreshed a track and delivers a variation.
The favorite icon stashed the track in your User Library.
The share icon copied a link to a public URL where anyone could listen.
The ellipsis included several additional options. You could open the riff in a dedicated window, save as audio or video, and split the audio into stems.
The Split out stems option would separate your track into vocals, drums, bass and "other" layers. Then download any individual audio layer and use them in a DAW.
Clicking on Open riff navigated to a full page view of the track. You could access many of the same features, but stem separation was no longer available and instead there was some control over the riff's visibility (public vs private).
That's pretty much how the V2 web app from October 2023 worked.
Now lets turn the clock back further in our Riffusion time machine, and go all the way back to the company's roots, long long ago, in December 2022.
Riffusion V1: The First AI Text-to-Music Generator
Riffusion's first site version debuted in December 2022.
It came in on the tail end of a year ripe with AI text-to-image generation. Midjourney and Dalle had become household words. OpenAI released ChatGPT3 and suddenly the world had access to conversational artificial intelligence.
Text prompts became a hot topic and the internet wanted to know when AI music generation would have its moment in the sun. Who would be the first to drop a service that converted descriptions of music into raw audio?
At the time, AudioCipher’s text-to-midi plugin was turning words into melodies and chord progressions, but it wasn't using a diffusion model. It gave users control over key signature, scale, chord extensions and rhythm automation. The app was intended for DAW musicians to beat writer's block.
Riffusion AI: Spectrogram Generator to Music
Riffusion made their first big move in December 2022 with a web app that created short music cues riffs at lightning speed. The creators, Seth Forsgren and Hayk Martiros, are indie programmers who started the app as a hobby project.
The website became a viral hit overnight.
For everyday people, it was one of those rare moments on the internet where you can send friends to a website and amaze them with something. So Riffusion was being shared virally by people who enjoyed the latest developments in AI.
However, the machine learning and genAI programming crowds celebrated the innovative underlying tech deployed by the company.
Riffusion V1 was built on top of a Stable Diffusion model called img2img. It trained on a multimodal dataset of labeled spectrograms. Each short audio clip was captured in an image and paired with a caption describing features like genre, instrument, speed, vibe, etc.
So this meant that when a user requested music in a particular style, Riffusion's model would generate new spectrograms in that similar style.
It interpolated and stitched the short clips together, sonifying the spectrograms and turning them into audio files.
Riffusion remained in this state for most of 2023.
Mid-2023: Google MusicLM, Meta MusicGen
A working demo of MusicLM didn’t reach the public until May 2023. It delivered higher quality audio and compositions, but lacked the exciting audio upload and style transfer features that had been promised by their demo page in January.
Meta swooped in a month later, making a press release in June about their own generative AI text-to-music model, called AudioCraft (MusicGen). They one upped Google by making AudioCraft open source, with a Github repo and API that developers could use to embed the service into their own interactive AI tools.
MusicGen was soon available on Hugging Face, and developers surfaced the same style transfer feature MusicLM had claimed to have but failed to deliver.
As of November 2023, MusicLM and MusicGen lack any AI voice features.
Late-2023: Stable Audio, Splash, and Chirp
Three months later, in September 2023, Stability AI announced a text-to-music web application called Stable Audio. Around this same time, companies like Chirp and Splash were surfaced with AI lyric-to-song generators.
Riffusion's V2 website and features, released in October 2023 around the time they announced their $4M seed round, put them firmly in the running with other rising stars. They fell behind in the first half of 2024 as Suno and Udio took the lead, but Riffusion's new mobile and desktop apps show a lot of promise.