Backpack Language Models
Backpack language models are a new neural architecture that marries strong modeling performance with an interface for interpretability and control. They learn multiple non-contextual sense vectors for each word, and represent a word in a sequence as a context-dependent, non-negative linear combination of sense vectors in this sequence. They can be used for Topic Controlled Generation, Mitigating Gender Bias and Knowledge Editing. Code, sense vectors, LM weights, and demos https://backpackmodels.science
In this video, I will talk about What are Backpack LMs? Embedding Quality and Topic Controlled Generation, Mitigating Gender Bias and Knowledge Editing using backpack LMs.
For more details, please look at https://arxiv.org/pdf/2305.16765.pdf
Hewitt, John, John Thickstun, Christopher D. Manning, and Percy Liang. "Backpack Language Models." arXiv preprint arXiv:2305.16765 (2023).