GPT Neo(GPT 3): Running On A CPU Vs A GPU
While you are able to run GPT Neo with just a CPU, do you want to? In this video, I explore how much time it takes to run the model on both the CPU and the GPU.
Notebook https://github.com/mallorbc/GPTNeo_no...
GPT Neo GitHub https://github.com/EleutherAI/gpt-neo (use the first release tag)
GPT Neo HuggingFace docs https://huggingface.co/transformers/m...
A useful article about transformer parameters https://huggingface.co/blog/how-to-ge...
Timestamps
0:00 - Intro
1:25 - Explanation of the tests
2:04 - Loading model onto GPU test
3:03 - Loading the model onto the CPU test
4:12 - Model inferences on the CPU test
5:19 - Model inferences on the GPU test
6:31 - Comparing the results and discussion
8:09 - Outro