Building style-aware neural MIDI synthesizers using simplified differentiable DSP appr. S. Grechin
Building style-aware neural MIDI synthesizers using simplified differentiable DSP approach
Authors: Sergey Grechin and RyanGroves (Infinite Album)
DRMN+16 Workshop talk, 21st December, QMUL.
Abstract— We explore how simplified differentiable DSP approach can be used to build realistic sounding MIDI-controllable monophonic synthesizers. The simplification involves directly using MIDI data as input to the DDSP decoder rather than continuous F0 and loudness curves. On top of that, we show how incorporating additional style-based and temporal channels can be used to imitate various aspects of performance and improve realism. We demonstrate the results by applying the approach to the task of modelling the sound of electric guitar. The presented results we reobtained with a model trained on less than 12 minutes of manually MIDI- annotated audio. The source code is released along with the prepared dataset.
Index Terms— Deep Learning, DDSP, virtual synthesizers, MIDI