Simple quantization of LLMs - a hands-on
One of my previous videos introduced the fundamentals of quantization. This video is about hands-on quantization of a small LLM model with Absolute Max and Zero Point quantization methods. A good grasp of these models help with learning advanced quantization method such as GPTQ, GGUF and AWQ.
This video is a precursor to videos on advanced quantization methods - both hands-on and theory.
Hope it's useful.
Notebook link: https://github.com/ai-bites/generativ...
MY KEY LINKS
YouTube: / @aibites
Twitter: / ai_bites
Patreon: / ai_bites
Github: https://github.com/ai-bites
WHO AM I?
I am a Machine Learning researcher/practitioner who has seen the grind of academia and start-ups equally. I started my career as a software engineer 15 years ago. Because of my love for Mathematics (coupled with a glimmer of luck), I graduated with a Master's in Computer Vision and Robotics in 2016 when the now happening AI revolution just started. Life has changed for the better ever since.
#machinelearning #deeplearning #aibites