Shiny Bald Guy - OpenAI Speech to Text + Metahuman Lipsync | UE5
Tried out Speech-to-Text with the OpenAI API. Made an AI-generated song with #Udio, grabbed the lyrics and timestamps with the OpenAI API, then pulled it together in #UnrealEngine with a #metahuman. Used the new lip-sync tool in Metahuman Performance for syncing the lips to the song.
It's pretty impressive, though the AI missed a few words here and there, and the lip-sync tool isn't yet quite on par with a real actor's performance.
Overall, it's been fun to see how these tools come together and make this kind of digital creation possible. 🔊
#unrealengine #metahuman #ai #udio #song #music #realtime