AI 3D Modeling: Why Cool Algorithms Fail to Profit?
3D modeling is widely used in the media, gaming, and education industries. In the past year, using AI to create 3D models, has gained much attention and many GenAI companies emerged.
I led a research team of several PhDs, finished a prototype, and also talked to quite a few game studios and artists. In this video, I’m gonna unfold the technical paths and share my views of the AI 3D space as a researcher and a founder.
Chapters:
0:00 Intro
0:32 3D modeling
1:20 AI 3D Algorithms
3:10 Industries - Gaming
3:53 Academia vs. Industry
5:18 AI Texturing Issues
5:29 Technical Challenges
6:54 AI 3D Startups
8:03 OpenAI & Nvidia
8:16 Startups Pivoted
8:52 Why startups struggle
9:46 Sloyd & Kaedim
10:15 Consumer / VR
10:43 Video Generation
11:15 Conclusion
#3dmodeling #ai
Recently, AI has gained traction in 3D asset creation, with numerous startups emerging in this space. A typical 3D asset includes mesh, texture, and UV mapping, requiring days to create manually, driving demand for AI solutions.
Various algorithms have been developed to assist 3D generation, inspired by AI techniques from 2D generation. Notable advances include 3DGAN, CLIP, and DreamFusion. My research team introduced EucliDreamer, which incorporates depth data to improve the speed and quality of 3D texturing.
The gaming industry represents the largest addressable market for AI-driven 3D assets, with a $10 billion value in 2022. However, there are challenges, such as AI's struggle with clean topology and inconsistent art styles. Gaming companies prefer converting images to 3D models for easier editing and style consistency. AI tools can aid early-stage asset creation, but the models are often too rough for commercial use, requiring significant human refinement.
Despite advancements, many startups have pivoted away from AI 3D due to technical hurdles, particularly the scarcity of high-quality 3D data and slow inference times. However, text-to-video models, such as OpenAI’s Sora, show promise in creating consistent 3D assets from video generation. The field is evolving, with success dependent on market fit and augmenting human creativity rather than replacing it.