there's no way to win (Modular & VCV Rack, Stable Diffusion/Deforum & TouchDesigner)
Audio reactive music video made with modular synth and AI. Thanks for checking it out! Details below:
The music was composed on modular synth and VCV Rack and then performed and recorded live. The AI animation part of the video was generated by a series of eight prompts using Automatic 1111 with the Deforum extension. Once rendered, this video gets processed in real time with TouchDesigner as the patch gets played to add the audio reactive elements and visual effects. The audio reactive elements are controlled by the music using followers for different voices in VCV Rack. The CV signals from the followers are sent to TouchDesigner with the trowaSoft OSC module where they animate various visual elements. Then the audio and video is captured together using OBS Studio as the patch is performed.
The first synth voice is the 0-Coast sequenced by the 0-Ctrl. This goes through the QPAS, Data Bender, and Desmodus Versio for reverb and back to the mixer in VCV Rack. This voice also makes the glitchy background chatter at certain points by setting the mix to full wet on the Data Bender.
The percussion that comes in next is the Blck_Noir being sequenced by the Eloquencer. The metallic bell is pulled from the module to add stereo movement and brought back together with the rest of the percussion in the mono section of Vortices. Then the mono mix runs through Guillotine for some distortion and feeds back to the stereo section of Vortices. The master mix of both sections then runs back to the mixer in VCV Rack. A copy of one channel of the master mix is also recorded into the Erica Synths Sample Drum where it is sliced up and run back to the mono section of Vortices to make the kind of break beat sounding fills.
The distorted bass voice is the Loquelic Iteritas, Manis Iteritas, and Ts-L mixed together with the X-Pan module. This mix goes through Ghost where is gets distortion, filtering, reverb, delay, and some ducking and then to the mixer in VCV Rack. Another copy of the voice get processing with Freak, Debriatus, and Plateau in VCV to add some extra density.
The chord stab is chainsaw through Ikarie and Mimeophone. It is sequenced with the Gate-Seq64 in VCV Rack because I ran out of hardware sequencing. The chord changes are initiated by the Eloquencer running back to VCV Rack which sends pitch information for the chords back to Chainsaw. A little convoluted, but I did it this way so that when the melody changes in modular the chord changes at the same time and they are linked together.
The patch is controlled manually by changing parts with the Eloquencer along with DivKid mutes and the nanoKontrol2 to bring in and out the voices and control the dynamics and modulation of everything.