AI agent breaks AI generated Minecraft
Earlier this week Decart and Etched released Oasis, a generative AI model that learns to generate a Minecraft-like game purely by learning from videos of Minecraft, with no coding.
Conventional Minecraft agents, such as Voyager, depend on language and API interfaces, making it challenging to operate in AI-generated versions of Minecraft. In contrast, this was pretty easy for us since some parts of our agent actually understand the world and play the game purely from vision. As a result, our agents can navigate and interact within these AI-generated landscapes. You can see the results compiled into a video.
Unexpectedly, the two AI models (our agent playing, and the Oasis model generating the world) tend to collapse into noise if you let them run for more than a few seconds. This is probably because both models were experiencing out-of-distribution data compared to what they were trained on. But, before that happens we think this shows a glimpse of a future, where AI’s imagine worlds and in those worlds both humans and AIs will participate and collaborate together.
For those interested in experiencing AI companions in the standard Minecraft environment, our AI agents actually play much better there. Download our app from elefant.gg and explore the future of gaming with AI today.