Improving Cross-Lingual Text Classification with Zero-shot Instance-Weighting (Paper Summary)
#textclassification #zeroshot #naturallanguageprocessing #crosslingual
This paper leverages the concept of instance-weighting for training a zero-shot cross-lingual text classification ml model. The idea revolves around giving more weight to the source instances that are similar to the target language and hence helping the model to learn the distribution on unseen language.
⏩ Abstract: Cross-lingual text classification (CLTC) is a challenging task made even harder still due to the lack of labeled data in low-resource languages. In this paper, we propose zero-shot instance-weighting, a general model-agnostic zero-shot learning framework for improving CLTC by leveraging source instance weighting. It adds a module on top of pre-trained language models for similarity computation of instance weights, thus aligning each source instance to the target language. During training, the framework utilizes gradient descent that is weighted by instance weights to update parameters. We evaluate this framework over seven target languages on three fundamental tasks and show its effectiveness and extensibility, by improving on F1 score up to 4% in single-source transfer and 8% in multi-source transfer. To the best of our knowledge, our method is the first to apply instance weighting in zero-shot CLTC. It is simple yet effective and easily extensible into multi-source transfer.
Sign-up for Email Subscription - https://forms.gle/duSwrYAGw6zUhoGf9
⏩ OUTLINE:
0:00 - Abstract and Introduction
02:21 - CLTC Framework illustration
05:50 - CLTC Loss
06:28 - Instance-weight calculation
⏩ Paper Title: Improving Cross-lingual Text Classification with Zero-shot Instance-Weighting
⏩ Paper: https://aclanthology.org/2021.repl4nl...
⏩ Author: Irene Li, Prithviraj Sen, Huaiyu Zhu, Yunyao Li, Dragomir Radev
⏩ Organisation: Yale University, IBM Research
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - / @techvizthedatascienceguy
**********************************************
If you want to support me financially which is totally optional and voluntary ❤️
You can consider buying me chai ( because I don't drink coffee :) ) at https://www.buymeacoffee.com/TechvizC...
❤️ Support using Paypal - https://www.paypal.com/paypalme/TechV...
**********************************************
⏩ Youtube - / techvizthedatascienceguy
⏩ LinkedIn - / prakhar21
⏩ Medium - / prakhar.mishra
⏩ GitHub - https://github.com/prakhar21
⏩ Twitter - / rattller
*********************************************
Tools I use for making videos :)
⏩ iPad - https://tinyurl.com/y39p6pwc
⏩ Apple Pencil - https://tinyurl.com/y5rk8txn
⏩ GoodNotes - https://tinyurl.com/y627cfsa
#techviz #datascienceguy #nlproc #machinelearning
About Me:
I am Prakhar Mishra and this channel is my passion project. I am currently pursuing my MS (by research) in Data Science. I have an industry work-ex of 3 years in the field of Data Science and Machine Learning with a particular focus on Natural Language Processing (NLP).