[2024 Best AI Paper] MemLong: Memory-Augmented Retrieval for Long Text Modeling
This video was created using https://paperspeech.com. If you’d like to create explainer videos for your own papers, please visit the website!
Title: MemLong: Memory-Augmented Retrieval for Long Text Modeling
Authors: Weijie Liu, Zecheng Tang, Juntao Li, Kehai Chen, Min Zhang
Abstract:
Recent advancements in Large Language Models (LLMs) have yielded remarkable
success across diverse fields. However, handling long contexts remains a
significant challenge for LLMs due to the quadratic time and space complexity
of attention mechanisms and the growing memory consumption of the key-value
cache during generation. This work introduces MemLong: Memory-Augmented
Retrieval for Long Text Generation, a method designed to enhance the
capabilities of long-context language modeling by utilizing an external
retriever for historical information retrieval. MemLong combines a
non-differentiable ``ret-mem'' module with a partially trainable decoder-only
language model and introduces a fine-grained, controllable retrieval attention
mechanism that leverages semantic-level relevant chunks. Comprehensive
evaluations on multiple long-context language modeling benchmarks demonstrate
that MemLong consistently outperforms other state-of-the-art LLMs. More
importantly, MemLong can extend the context length on a single 3090 GPU from 4k
up to 80k. Our code is available at https://github.com/Bui1dMySea/MemLong