We will learn about how to crawl a documentation website. How to detect if code exists in a given file using gpt-3.5-turbo-16k, extract self sufficient code blocks along with generated explanations and create a dataset of the entire Langchain documentation. Then we will embed it and search over it using similarity search and let GPT-4 generate entire langchain applications from simple descriptions
Extracting self sufficient code chunks cost $75 in API usage. You can download them below ⬇️⬇️⬇️
Code files with extracted code dataset and Vectordb download:
/ 87972378
Download code files without extracted code and embeddings csv:
/ 87972389
Source code for AUTO AGI: / code-files-for-87530987
Search 140+ echohive videos and code download links:
https://www.echohive.live/
Chat with us on Discord:
/ discord
Office hours with me: / currently-in-87598161
LLM Paper Summaries:
https://llmpapers.up.railway.app/
Try the GPT-4 Auto Coder app:
https://gpt4autocoder.streamlit.app/
CHAPTERS:
00:00 Intro
00:40 DEMO
02:04 Code download
02:19 Overview of files
05:11 Documentation crawler code review
07:11 Code Extractor code review
15:51 Creating embeddings file code review
19:32 Search and chat file code review
#langchain #langchaintutorial #gpt4 #openai #openaiembeddings
#vectordb #artificialintelligence #pythonprojects