Unlike Pluto - Sinking Sand (Lyrics)
Animated AI background video generated in Deforum Stable Diffusion. Compositing, lyrics and audio reactive effects edited in After Effects.
Music 🎵
Unlike Pluto - Sinking Sand
Playlists 🤘
Lyric Videos: • Lyric Videos
Lyrics 🎶
1. Finding a couple of fitting prompts, often with the help of ChatGPT. I've asked it to generate prompts based on the song lyrics I give it. Then just a lot of testing and experimentation with keywords in Stable Diffusion, to find prompts that vibe with the song.
2. Feed the song to the chigozie audio keyframe generator, matching the fps with my project (I do 25) and entering a suitable function. I use something like "1.02 + x^5" for zoom and "0.60 - x^6" for strength schedule, but I often play around with them to find different results.
3. Entering the keyframes and prompt into the Automatic1111 extension Deforum and generating an animation (I usually do this 4 times with different settings/prompts to get more footage to work with later)
4. Editing the generated sequences in Premiere and then adding lyrics/effects in After Effects. I use the multicam feature in Premiere to cut between the generated animations, resulting in a more dynamic background with mostly the best parts. I create the lyrics in After Effects and also add a bunch of effects like additional camera movements, shakes, scratches, particles, blur and chromatic aberration. Most of these are made to react to the music through generated audio keyframes.
#lyrics #music