Building a Text Generator With LSTM, From Scratch!: PyTorch Deep Learning Tutorial
TIMESTAMPS
0:00 - Introduction
0:28 - Overview
1:44 - Next token prediction and its role in language models (LLMs).
2:27 - Preparing data: Training a SentencePiece tokenizer and custom dataset creation.
4:36 - Training the tokenizer model and vocabulary creation.
6:24 - Incorporating tokenizer into dataset transforms and data loading.
9:24 - Implementing LSTM architecture for token prediction.
12:26 - Description of model architecture and training process.
15:05 - Training the model and analyzing prediction entropy.
19:52 - Generating text using trained model with sample prompts.
25:43 - Evaluating generated text against original articles from test set.
27:53 - Adjusting temperature parameter for controlling randomness in text generation.
In this video I show how we can use the concepts of LSTM many-to-many prediction together with text data to produce a basic text generator trained on news articles!
Discord Server:
/ discord
Donations
https://www.buymeacoffee.com/lukeditria
The corresponding code is available here! (Section12)
https://github.com/LukeDitria/pytorch...