Building a Real-Time Voice AI NPC in Unreal Engine 5 (C++, Blueprints & REST APIs)
This isn't a scripted chatbot.
Players can speak or type naturally, and the NPC responds in real time using Google Gemini for reasoning and ElevenLabs for voice synthesis. The AI personality can be changed instantly through Blueprints without rewriting dialogue.
Behind the scenes, we built a complete end-to-end pipeline featuring:
• REST API integration with JSON serialization
• Custom C++ Base64 decoding and disk I/O
• Dynamic timestamp-based file versioning
• Asynchronous runtime audio importing
• Low-latency speech playback
• UE5 C++ & Blueprints hybrid architecture
👥 PROJECT CREATORS:
• Golla Sukumar – System Architecture, C++ & UE5 Engineering
• Sai Kiran K – Original Concept, AI Design & Development
💻 TECH STACK:
• Unreal Engine 5 (C++ & Blueprints)
• Google Gemini API
• ElevenLabs API (wav_24000)
• VaRest Plugin
• Runtime Audio Importer Plugin
We're constantly exploring the future of AI-powered games and interactive characters. Feedback and suggestions are always welcome!
⏱️ TIMESTAMPS:
0:00 - Live System Demonstration (Real-Time Dialogue)
0:22 - Explaining the Architecture
1:28 - Versatility of the AI Personality
🔗 LINKS & CONTACT:
• Golla Sukumar : / golla-sukumar-44729a365 | [email protected]
• Sai Kiran K : / sai-kiran-k-2a20b9287 |
[email protected]
#unrealengine5 #ue5 #gamedev #artificialintelligence #conversationalai #cplusplus #googlegemini #elevenlabs #ai #buildinpublic