ImageBind from Meta AI - One Embedding Space To Bind Them All
ImageBind is the first AI model that is able to make sense out of six different types of data, bringing AI a step closer to how humans are sensing the environment.
If you enjoy this video then please subscribe to the channel and hit the like button to support the creation of more similar educational content.
In this video we will explain what ImageBind model is, then we'll explore few interesting usages of ImageBind such as Cross-Modal Retrieval and Audio-to-Image generation using DALLE-2.
We'll then go through how Meta AI were able to create ImageBind, by gathering matching pairs of images and samples from the other modalities, and then use contrastive learning to move matching samples closer together in one embedding space.
ImageBind is using CLIP text and image encoders and do not change them at all during training, so the other modalities are added to the embedding space created by CLIP, which makes it easier to replace CLIP embedding in other models and use ImageBind embedding instead, for example in order to be able to move from text to image generation, to audio to image generation.
Blog post - https://aipapersacademy.com/imagebind/
GitHub repo - https://github.com/facebookresearch/I...
Arxiv paper - https://arxiv.org/abs/2305.05665
Chapters:
0:00 Introducing ImageBind
0:34 What is ImageBind
1:34 Cross-Modal Retrieval
2:53 Audio to Image Generation
3:51 How ImageBind Was Created