Multi-modal RAG with LlamaIndex and
Multi-Modal LLM using @Google Gemini model for image understanding and build Retrieval Augmented Generation with LlamaIndex. In this video, we show how to use Google's Gemini Vision models for image understanding.
We will also in the end will be using Gemini & LlamaIndex to build a simple Retrieval Augmented Generation Flow for a small Google Maps restaurant dataset.
Thank you for watching. Please like share and subscribe for more such videos. Happy Learning
0:00 Introduction
0:29 Tasks to be covered
1:09 Importing libraries
2:06 Use Gemini to understand Images from URLs
3:10 Initialize GeminiMultiModal and Load Images from URLs
5:04 Complete a prompt with bunch of images
5:57 Stream complete a prompt with a bunch of images
7:01 Compare images
9:19 Create structured output from an image with Pydantic and Gemini
12:24 Create your structured class
13:47 Custom method to generate structured response
21:10 Build Multi model RAG for Restaurant recommendation
24:35 Create your vector store index
26:38 Using index as a query engine o get recommendations
29:02 Conclusion
29:13 Like Share and Subscribe
Google colab notebook - https://colab.research.google.com/gis...
Gemini API key - https://aistudio.google.com/
LlamaIndex - https://www.llamaindex.ai/