Real-Time AI Conversations & Facial Animation for MetaHumans | Convai Unreal Engine Tutorial
In this tutorial, we show how to integrate Convai with Metahuman avatars in Unreal Engine, enabling natural, real-time conversations with accurate lip sync and facial animation.
Using Convai’s updated @UnrealEngine plugin, your Metahuman can listen, understand, and respond naturally through voice, while Convai’s in-house neural animation system, NeuroSync, processes the live audio stream in real time. This drives highly accurate lip movements and facial expressions using MetaHuman’s 250+ blend shapes, resulting in expressive, low-latency conversational AI characters.
This video focuses specifically on AI-powered Metahumans in Unreal Engine: from connecting a Convai character brain to a Metahuman, to configuring speech input, facial animation and more. By the end, you’ll have a Metahuman that can hold natural spoken conversations with believable facial animation, suitable for NPCs, virtual humans, training simulations, XR experiences, and interactive storytelling.
Unreal Engine Set-Up Documentation: https://docs.convai.com/api-docs/plug...
Convai Plugin: https://www.fab.com/listings/ba3145af...
AI Characters with Streaming Vision Input and Handsfree Interactions | Convai Unreal Engine Tutorial: • Video
Read the full blog here: https://home.convai.com/blog/real-tim...
Time Stamps:
00:00 Intro & Demo: Stargazing AI NPC
00:24 Tutorial Overview
01:23 Enable Convai Plugin
01:59 Import MetaHuman via Quixel Bridge
02:28 Add Chatbot Component (The Brain)
02:41 Get Character ID from Convai
03:10 Add FaceSync Component (The Animation)
03:29 Enable Lipsync for Metahuman Blendshapes
03:35 Configure Animation Blueprints
03:52 Add Components for Enabling Speech
04:26 Set Up Push-to-talk / Handsfree Interaction
04:38 Final Demo
05:58 Outro & Community Info
Sign up at https://convai.com to try it out. And learn more about how Convai’s plugins can be used across the board.
Have questions or need assistance?
Visit the Convai Developer Forum – our dedicated community and support team are here to help with bug reports and technical queries. Get the support you need and connect with other developers to enhance your experience.
Convai Developer Forum: https://forum.convai.com/
Follow us on your favorite platforms to stay in touch:
LinkedIn: / convaitech
Reddit: / convai
Twitter: / convaitech
Instagram: / conv.ai
Facebook: / convaiofficial
TikTok: / conv.ai
ArtStation: https://www.artstation.com/convai