Google Flan: Finetuned Language Models are zero shot learners
Google FLAN is an instruction tuned model which is tuned on 137B LaMDA-PT using 60 NLP datasets.
0-shot FLAN is better than 0-shot LaMDA-PT.
0-shot FLAN is better than 0-shot 175B GPT-3 on 20/25 datasets.
0-shot FLAN is better than few-shot GPT-3 on NLI (ANLI, RTE), Reading comprehension (BoolQ, OpenbookQA), AI2-ARC, and StoryCloze.
0-shot FLAN is better than 0-shot GLaM on 13/19 datasets and 1-shot GLaM on 11/19 datasets.
Performance on unseen tasks improves with the number of instruction tuning task clusters, and model scale.
Instruction tuning can be combined with other prompting methods such as few-shot prompting and prompt tuning.
Here is the agenda for this video:
00:00:00 What is FLAN?
00:00:00 What are the tasks and templates used to train FLAN?
00:00:00 How does 0-shot FLAN compare with few-shot GPT3?
00:00:00 What factors are important for zero-shot instruction tuning?
00:00:00 What is the impact of few shot examples and prompt tuning on instruction tuning?
00:00:00 Some FLAN output examples
For more details, please look at https://arxiv.org/pdf/2109.01652.pdf and https://ai.googleblog.com/2021/10/int...
Wei, Jason, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. "Finetuned Language Models are Zero-Shot Learners." In International Conference on Learning Representations. 2022.