DialoGPT: Generative Training for Conversational Response Generation (Research Paper Walkthrough)
#dialogpt #chatbot #gpt
This paper explains DialoGPT, an autoregressive (AR) language model that uses multi-layer transformer as model architecture for modeling response generation for open-domain conversational agents.
⏩ Abstract: We present a large, tunable neural conversational response generation model, DIALOGPT (dialogue generative pre-trained transformer). Trained on 147M conversation-like exchanges extracted from Reddit comment chains over a period spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch transformer to attain a performance close to human both in terms of automatic and human evaluation in single-turn dialogue settings. We show that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems. The pre-trained model and
training pipeline are publicly released to facilitate research into neural response generation
and the development of more intelligent opendomain dialogue systems.
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - / @techvizthedatascienceguy
⏩ OUTLINE:
0:00 - Background and Overview
2:50 - Abstract
5:14 - Dataset and Pre-processing steps
8:17 - Model Architecture
10:46 - Mutual Information Maximization
⏩ Paper Title: DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
⏩ Paper: https://www.aclweb.org/anthology/2020...
⏩ Code: https://github.com/microsoft/DialoGPT
⏩ Author: Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
⏩ Organisation: Microsoft Corporation, Redmond, WA, USA
⏩ IMPORTANT LINKS
DialoGPT Blog with Sample Code - https://towardsdatascience.com/dialog...
ACL Papers Explanations - • Unsupervised Multi-Document Summariza...
T5 Paper - • T5: Exploring Limits of Transfer Lear...
Nucleus Sampling Paper - • Nucleus Sampling: The Curious Case of...
Enjoy reading articles? then consider subscribing to Medium membership, it just 5$ a month for unlimited access to all free/paid content. Subscribe now - / membership
*********************************************
If you want to support me financially which totally optional and voluntary :) ❤️
You can consider buying me chai ( because i don't drink coffee :) ) at https://www.buymeacoffee.com/TechvizC...
*********************************************
⏩ Youtube - / techvizthedatascienceguy
⏩ Blog - https://prakhartechviz.blogspot.com
⏩ LinkedIn - / prakhar21
⏩ Medium - / prakhar.mishra
⏩ GitHub - https://github.com/prakhar21
⏩ Twitter - / rattller
*********************************************
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - / @techvizthedatascienceguy
Tools I use for making videos :)
⏩ iPad - https://tinyurl.com/y39p6pwc
⏩ Apple Pencil - https://tinyurl.com/y5rk8txn
⏩ GoodNotes - https://tinyurl.com/y627cfsa
#techviz #datascienceguy #nlproc #researchpaper #naturallanguageprocessing #transformers #nlproc