Fine tuning GPT-OSS 20b open source model from OpenAI
In this video, let's look into fine-tuning the open-source 20b parameters GPT-OSS model. We will use the HugginFace ecosystem for this. It includes the transformers, peft, dataset, and the related libraries.
The fine-tuning process is mostly following a boilerplate code other than that we need to be wary of what datasets we use, what hardware resources we need, and at times, the parameters we need to set before starting the training.
Hope it's useful!
Github Notebook: https://github.com/ai-bites/generativ... (look for gpt-oss-ft.ipynb)
Fine-tuned model - https://huggingface.co/ai-bites/gpt-o...
GPT-OSS - https://openai.com/index/introducing-...
HF dataset - https://huggingface.co/datasets/Huggi...
OpenAI's cookbook - https://cookbook.openai.com/articles/...
AI BITES KEY LINKS
Website: https://www.ai-bites.net
YouTube: / @aibites
Twitter: / ai_bites
Patreon: / ai_bites
Github: https://github.com/ai-bites