Fully local RAG agents with Llama 3.1
With the release of Llama3.1, it's increasingly possible to build agents that run reliably and locally (e.g., on your laptop). Here, we show to how build reliable local agents using LangGraph and Llama3.1-8b from scratch. We build a simple corrective RAG agent w/ Llama3.1-8b, and compare its performance to larger models llama3-70b, gpt4-o. We test our Llama3.1-8b agent on a corrective RAG challenge, and show performance and latency versus a few competing models. On our small / toy challenge, Llama3.1-8b performs on par w/ much larger models w/ only slightly increased latency. Overall, Llama3.1-8b model is a strong option for local execution and pairs well with LangGraph to implement agentic workflows .
Blog post:
https://ai.meta.com/blog/meta-llama-3-1/
Ollama:
https://ollama.com/library/llama3.1
Code:
https://github.com/langchain-ai/langg...