Virtual Piano Player Animated by AI!
Video created with Concert Creator: https://concertcreator.ai
A demo of AI that can take an audio recording or MIDI file and turn it into a full 3d animation of a person playing the piano!
The AI is generating all the movements in the video including correct finger placement and hand movements for the piano. This is not simple IK, it can only play music that a human being could actually play (so no 4 handed or 6 fingered music ;-)
Unfortunately, the software only exports renderings and not animations that we could use with our own characters and scenes, but maybe if enough people show interest the people at Concert Creator will offer it, so ask!
The song is Maple Leaf Rag by Scott Joplin from 1899 which is now copyright free. This performance is based on the original sheet music and is entirely synthetic including the audio.
I believe this is being rendered by the Unity engine but I'm not positive. This is a path traced version, medium quality from the RTX version of Concert Creator. It took about 1 second per frame to render on an NVIDIA 3090 card.
#virtualproduction #AI #unity