Running GPT-NeoX-20B With Hugging Face
GPT-NeoX-20B has been added to Hugging Face! But how does one run this super large model when you need 40GB+ of Vram? This video goes over the code used to load and split these large language models over multiple devices in order to run them.
Github: https://github.com/mallorbc/GPTNeoX20...
Discord: / discord
Donations(if you want):
Ethereum:
4CE913643909Fa3168297cC2857C0aDdAB389Ad8
Monero: 45mqN96o5JZZhwVTZHHiQJeyhp3WndiPC44hdrAmWeDGeCmaC1c45gTGh5eDUtEhx3JDGbsAnsD3VBXKdiorUhusUydLG22
Timestamps
00:00 - Intro
00:18 - Requirements
01:42 - Cloning And Env Setup
02:47 - Running Options
03:24 - Looking At The Code
06:33 - First Time Running Code
08:04 - Running The Second Time
09:10 - Outro And Work With Me!