How To Fine-tune The Llama 1 Models(GPT3 Alternative)
The LLaMA models have impressive performance despite their relatively smaller size, with the 13B model being even better than GPT3!
In this video, I go over how you can make the models even more powerful, by finetuning them on your own dataset!
Github: https://github.com/mallorbc/Finetune_...
Model Link: https://huggingface.co/decapoda-resea...
LLaMA PR: https://github.com/huggingface/transf...
LLaMA Paper: https://arxiv.org/pdf/2302.13971.pdf
GPT3 Paper: https://arxiv.org/pdf/2005.14165.pdf
Discord: / discord
#ai #chatgpt #docker #gpt3 #machinelearning #nlp #llama #gpt4 #wandb #llm
Timestamps
00:00 - Intro
00:16 - Model Metrics And Explanation
01:24 - Github Repo
01:39 - Finetuning Process Differences
02:36 - Setup Walkthrough
04:23 - Running Docker Image
06:00 - Looking At Run Flags
08:05 - Getting The Model Weights
10:11 - 3090 Server Performance
11:16 - A100 Server Performance
11:40 - WandB Loss Graphs
12:09 - Finetuned Model Inference
13:07 - Outro