LLMOps (LLM Bootcamp)
In this video, Josh gives a tour of the emerging discipline of LLMOps: principles and practices for continuous improvement of large language model-powered applications.
Comparing and evaluating open source and proprietary models
Workflows and tools for iteration and prompt management
Principles for applying test-driven development to LLMs
Download slides and view lecture notes: https://fullstackdeeplearning.com/llm...
Intro and outro music made with Riffusion: https://github.com/riffusion/riffusion
Watch the rest of the LLM Bootcamp videos here: • LLM Bootcamp - Spring 2023
00:00 Why LLMOps?
01:55 Choosing your base LLM
04:20 Proprietary LLMs
09:15 Open-source LLMs
14:45 Iteration and prompt management
22:35 Testing LLMs: Why and why is it hard?
28:15 Testing LLMs: What works?
35:05 Evaluation metrics for LLMs
37:40 Deployment and monitoring
44:35 Test-driven development for LLMs