I built a GPT model in 10 hours
Kapathy’s tutorial: • Let's build GPT: from scratch, in code, sp...
Link to the Colab (make a copy of it to fill it in): https://colab.research.google.com/dri...
GPT theory videos:
3blue1brown’s super helpful videos on this: • Attention in transformers, step-by-step | ... and • Transformers, the tech behind LLMs | Deep ...
Algorithmic Simplicity’s video give great intuition: • Transformer Neural Networks Derived from S...
PyTorch nn.Module tutorial: https://pytorch.org/tutorials/beginne...
When I didn’t understand things, I mostly just asked ChatGPT to explain it to me. The 4o model has just become freely available, so give it a go!