Generative AI music companies have been promising to "unlock creativity" since late 2022. A few of the big ones, namely Suno and Udio, have pulled this off in a way that would have seemed impossible just a few years ago.
The value of text-to-music apps like Suno are obvious for non-musicians looking to express their imagination with limited skills and experience.
Musicians, on the other hand, have taken the time to learn scales and chords, navigate a DAW, and pick sounds that express what they feel inside. Song craft is a labor of love. Most of us want to automate the boring parts of engineering or get a little help imagining the next section of a song.
In this article, I'm going to share a technique that harnesses the power of text-to-music generation and AI video generators, without giving up your creative agency in the DAW. You'll be able to express the mental imagery that's inspired your track and create high quality visual content to go along with it.
FrogSynth: How To Create Music with AudioCipher
In the video above, we use AudioCipher's text-to-MIDI generator to create a short melody, two bass lines, and a chord progression. I'll explain that process below in a step by step method that's easy to follow.
Step 1. Choose a word or idea that inspires you
We started with the idea of FrogSynth, a parody genre invented by the AI music group Dadabots and featured in their song Genre Canon. It sounded like a funny concept, creating synthesizers that sound like bullfrogs. This idea quickly turned into frogs in outer space, and we thought that would make for a funny video.
Step 2. Open your DAW and pick some MIDI instruments
Next, we opened our DAW and created an empty MIDI track. I searched for the word "frog" in Omnisphere and found a bass instrument that sounded perfect. With that instrument armed, it was time to get to work.
Step 3. Experiment with text-to-MIDI in AudioCipher (Frog, Toad)
I opened AudioCipher on a separate MIDI track and tuned it to the key of F minor. This was a bit of a joke, since we were using the word "FROG". The minor key seemed right for a swamp creature. Also, while we're doing frogs in space, we went ahead and cranked up the tempo to 140 BPM.
The song needed something more, so I found an ethereal patch to represent outer space and switch AudioCipher to chord generation mode. I used the word "Toad" and moved the rhythm slider to whole notes, with chord inversions and note joining turned on. This created a nice progression that I decided to keep.
Lastly, for some variation, I returned to quarter-note mode and created a second bass line variation with the word "Toad", shifting the octaves of the first and last note to match the shape of the "Frog" bass line.
Step 4. Expand on the arrangement - Aim for 30-90 seconds of music
Our goal with this exercise wasn't to finish a song. Instead, it's about capturing our mental imagery and express it with a video. Most social media clips are only viewed for 30-60 seconds anyway, so we may as well keep it short and sweet.
I added a drum n' bass section, elaborated on the bass line to make it more rhythmic, and used "frog" for a fast top-melody during the middle section.
Step 5. Generating AI images that match the theme
There are plenty of AI image generation tools to choose from these days. I went with Midjourney for this video. You can see my prompt above and will recognize it from the opening scene of the video.
Pro tip: If there's a certain style you're to achieve, you can use character reference and style reference flags in Midjourney to capture that.
Step 6. Bringing your images to life with AI video tools
The final step is to bring that imagery to live with an AI video generator. My personal favorite is Neural frames because it's the only audio reactive tool creating frame by frame animations like this.
If you want to compare your options, we have a full length writeup on the most popular AI music video generators here on our blog. You can also go for non-reactive tools like Kling AI, RunwayML, Pika, Luma, or any number of others.
ChipDragon: Saving your ideas in the MIDI Vault
Now that we've covered the technique, I'll share a few more examples that might inspire you. This second demo was created using the word "dragon" in at 120 BPM and switching over to AudioCipher's chord generation mode.
In the second half of the video, I showcase the MIDI Vault workflow. The most important MIDI and audio files from the project are bounced and then moved into an AudioCipher card. This way I can easily add meta tags and access those ideas in the future, without digging through my DAW project files or bloat folders.
Combining AudioCipher with other MIDI generators
In this third demo, we show how AudioCipher's text-to-MIDI generator can be used with other apps. We collaborated with indie AI video artist Tony Matos to build out a whole world around the initial idea of "waterfalls" and floating islands.
A friend of mine is working on an Ableton Max4Live patch that turns text into MIDI, so we used that as a starting point to generate a two chord phrase. I completed the phrase with two additional chords and used AudioCipher to come up with countermelodies using the words "Waterfall", "Relic", and "Clouds"
Those three words were handed off to the video artist, who incorporate them into the mystical off-world imagery that he created.
You can find a second "snowflake" theme below that used the same Ableton patch with AudioCipher to evoke the feelings of a winter wonderland.
Ready to get in there and try this out yourself? Visit our website to learn more about the app and pick up a copy today. If you already own a copy and need to upgrade to the latest version, contact us directly for your custom discount code.
You can find detailed tutorials explaining how to get the most out of the app over at our FAQ page. Check out the free PDF at the top of page for a walkthrough on all of our features.