Recursive Language Models: Why Bigger Context Windows Fail
You can hand an AI model 200,000 tokens, the answer sitting right there inside, and it still gets the question wrong. That gap between fitting information in and actually using it is the whole story, and it has a name: context rot.
This video breaks down Recursive Language Models (RLMs), a new approach from MIT that flips the usual playbook. Instead of stuffing everything into the model's context window, RLMs keep the data outside in a coding workspace, like a Python environment, and let the model search, filter, count, and split only the pieces it actually needs. We walk through it with a real example: digging through 500,000 customer support chats to find out why refunds suddenly spiked, one narrowing step at a time.
You will also see the simple rule that makes it work: use code for exact counting and math, and use a language model only for meaning. Then the results: on the hard long-context benchmark OOLONG, an RLM running the smaller, cheaper GPT-5-mini beat full-size GPT-5 by about 114 percent overall, stayed roughly 49 percent ahead even near 263,000 tokens, and kept working past 10 million tokens where normal models fall apart. Plus the honest catch: recursion can get slow and expensive on easy questions, so use the simplest tool that works.
0:00 - The 200k paradox
0:27 - Context rot
0:58 - Keep it outside
1:27 - Filter by date
1:55 - Search and split
2:25 - The findings
2:52 - Code or meaning
3:16 - Does it work
3:54 - Small desk, big warehouse
If this made long context click for you, subscribe for more plain-English AI explainers, hit the bell, and drop a comment: what would you stop feeding your model if it could just go and fetch?
RLM ContextRot AI LLM LongContext MachineLearning GPT5 AIExplained