Simple text generation
This video is the second part of the "Probability-based random text generation" video of the "Computer recreations" playlist. You should watch that video first to understand this one.
You'll see how random text can be generated in a more efficient way than using n-dimensions matrices. Using Python `dict` objects enables you to store the frequency occurrences of sub-strings in a way that uses much less memory. This is especially useful for bigger orders.
If you use an n-dimension matrix (n = order) the memory usage growth is exponential.
Links
Original video: • Probability based random text generation
Source code: https://codeberg.org/frnmst/solve-com...
CHAPTERS
0:00 Intro
0:14 Example of probability-based randomly generated texts: having a look at the results for different orders and checking if the text already exists using PizzaGPT
3:09 List of steps to generate a text of this kind
6:40 Text classification intro: using dicts is much more efficient than using the matrix, as I did in the previous video. Also: some more remarks and details on the other steps
7:25 The text classification step in detail: using a subset of the big text to show how it's classified
8:56 Text classification with the matrix (old method): memory usage
10:03 Text classification using dicts: much more efficient memory wise
11:20 Text generation phase: adapting the older implementation to be used with a dict instead of a matrix
13:15 Overview of the `__main__` function in the script
13:51 Generating new texts from different URLs
15:15 Feeding the generated text to the script two times: `generate(generate(generate()))`
16:08 Outro
#computerrecreations #python #textgeneration #solvecomputerscience