TARS: Task-Aware Representation of Sentences for Generic Text Classification (NLP Paper Summary)
#textclassification #zeroshot #naturallanguageprocessing
State-of-the-art approaches for text classification leverage a transformer architecture with a linear layer on top that outputs a class distribution for a given prediction problem. While effective, this approach suffers from conceptual limitations that affect its utility in few-shot or zero-shot transfer learning scenarios. First, the number of classes to predict needs to be pre-defined. In a transfer learning setting, in which new classes are added to an already trained classifier, all information contained in a linear layer is therefore discarded, and a new layer is trained from scratch. Second, this approach only learns the semantics of classes implicitly from training examples, as opposed to leveraging the explicit semantic information provided by the natural language names of the classes. For instance, a classifier trained to predict the topics of news articles might have classes like “business” or “sports” that themselves carry semantic information. Extending a classifier to predict a new class named “politics” with only a handful of training examples would benefit from both leveraging the semantic information in the name of a new class and using the information contained in the already trained linear layer. This paper presents a novel formulation of text classification that addresses these limitations. It imbues the notion of the task at hand into the transformer model itself by factorizing arbitrary classification problems into a generic binary classification problem. We present experiments in few-shot and zero-shot transfer learning that show that our approach significantly outperforms previous approaches on small training data and can even learn to predict new classes with no training examples at all.
Sign-up for Email Subscription - https://forms.gle/duSwrYAGw6zUhoGf9
⏩ OUTLINE:
0:00 - Abstract and Background
05:19 - Universal Binary Text Classification Formulation
08:32 - TARS Model Architecture
09:54 - Training and Prediction
10:33 - Model Transfer and Computational Complexity
11:32 - Experiments and Results (Few-shot and Zero-shot)
14:42 - Cross-Domain Transfer Results
⏩ Paper Title: Task-Aware Representation of Sentences for Generic Text Classification
⏩ Paper: https://aclanthology.org/2020.coling-...
⏩ Code: https://github.com/flairNLP/flair
⏩ Author: Kishaloy Halder, Alan Akbik, Josip Krapac, Roland Vollgraf
⏩ Organisation: Zalando SE, Humboldt-Universitat zu Berlin
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - / @techvizthedatascienceguy
**********************************************
If you want to support me financially which is totally optional and voluntary ❤️
You can consider buying me chai ( because I don't drink coffee :) ) at https://www.buymeacoffee.com/TechvizC...
❤️ Support using Paypal - https://www.paypal.com/paypalme/TechV...
**********************************************
⏩ Youtube - / techvizthedatascienceguy
⏩ LinkedIn - / prakhar21
⏩ Medium - / prakhar.mishra
⏩ GitHub - https://github.com/prakhar21
⏩ Twitter - / rattller
*********************************************
Tools I use for making videos :)
⏩ iPad - https://tinyurl.com/y39p6pwc
⏩ Apple Pencil - https://tinyurl.com/y5rk8txn
⏩ GoodNotes - https://tinyurl.com/y627cfsa
#techviz #datascienceguy #researchpaper #fewshotlearning #zeroshotlearning #representationlearning #nlproc #machinelearning
About Me:
I am Prakhar Mishra and this channel is my passion project. I am currently pursuing my MS (by research) in Data Science. I have an industry work-ex of 3 years in the field of Data Science and Machine Learning with a particular focus on Natural Language Processing (NLP).