The Reversal Curse for LLMs
Reversal Curse is a surprising failure of generalization in auto-regressive LLMs. If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". E.g., if model is trained on "Olaf Scholz was the ninth Chancellor of Germany", it will not automatically be able to answer the question, "Who was the ninth Chancellor of Germany?". Moreover, the likelihood of the correct answer ("Olaf Scholz") will not be higher than for a random name. The Reversal Curse does not apply for in-context learning though.
In this video, I will talk about the following: What is the reversal curse for LLMs? Reversing descriptions of fictitious celebrities, Reversal Curse for real-world knowledge.
For more details, please look at https://arxiv.org/pdf/2309.12288.pdf
Berglund, Lukas; Tong, Meg; Kaufmann, Max; Balesni, Mikita; Cooper Stickland, Asa; Korbak, Tomasz; Evans, Owain. The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A". arXiv:2309.12288.