Leveraging BERT for Extractive Text Summarization on Lectures (Research Paper Walkthrough)
#bert #textsummarization #researchpaperwalkthrough #nlp
Automatic summarization is the process of shortening a set of data computationally, to create a subset that represents the most important or relevant information within the original content. Extractive summarization can be seen as the task of ranking and scoring sentences in the document based on certain metrics and then picking top-k sentences as the representative summary.
✔️ Automatic Python code generation from Spreadsheet - Mito - https://bit.ly/3eo4Wwi
⏩ Abstract: In the last two decades, automatic extractive text summarization on lectures has demonstrated to be a useful tool for collecting key phrases and sentences that best represent the content. However, many current approaches utilize dated approaches, producing sub-par outputs or requiring several hours of manual tuning to produce meaningful results. Recently, new machine learning architectures have provided mechanisms for extractive summarization through the clustering of output embeddings from deep learning models. This paper reports on the project called Lecture Summarization Service, a python based RESTful service that utilizes the BERT model for text embeddings and KMeans clustering to identify sentences closes to the centroid for summary selection. The purpose of the service was to provide students a utility that could summarize lecture content, based on their desired number of sentences. On top of the summary work, the service also includes lecture and summary management, storing content on the cloud which can be used for collaboration. While the results of utilizing BERT for extractive summarization were promising, there were still areas where the model struggled, providing feature research opportunities for further improvement.
⏩ OUTLINE:
0:00 - Quick Refresher on Text Summarization and BERT
6:20 - Intro and Overview
9:35 - Method
17:12 - Ensemble Models
18:48 - My thoughts and takeaways
⏩ Paper: https://arxiv.org/abs/1906.04165
⏩ Author: Derek Miller
⏩ Organisation: Georgia Institute of Technology, Atlanta, Georgia
⏩ IMPORTANT LINKS:
BERT - https://arxiv.org/abs/1810.04805
TextRank - https://web.eecs.umich.edu/~mihalcea/...
Text Summarization Survey - https://arxiv.org/abs/1707.02268
Extractive Text Summarization Survey - https://ieeexplore.ieee.org/document/...
In case you like reading instead of consuming video content, make sure to check out my Blog: / cbrr2skw0fb
Full Playlist on BERT usecases in NLP: • Text Summarization of COVID-19 Medical Art...
Full Playlist on Text Data Augmentation Techniques: • Data Augmentation using Pre-trained Transf...
Full Playlist on Text Summarization: • Text Summarization of COVID-19 Medical Art...
Full Playlist on Machine Learning with Graphs: • DEEPWALK: Online Learning of Social Repres...
Full Playlist on Evaluating NLG Systems: • Evaluation of Text Generation: A Survey | ...
*********************************************
If you want to support me financially which totally optional and voluntary :) ❤️
You can consider buying me chai ( because i don't drink coffee :) ) at https://www.buymeacoffee.com/TechvizC...
*********************************************
⏩ Youtube - / @techvizthedatascienceguy
⏩ Blog - https://prakhartechviz.blogspot.com
⏩ LinkedIn - / prakhar21
⏩ Medium - / prakhar.mishra
⏩ GitHub - https://github.com/prakhar21
*********************************************
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - / @techvizthedatascienceguy
Tools I use for making videos :)
⏩ iPad - https://tinyurl.com/y39p6pwc
⏩ Apple Pencil - https://tinyurl.com/y5rk8txn
⏩ GoodNotes - https://tinyurl.com/y627cfsa
#techviz #datascienceguy #transformers #pytorch #huggingface #naturallanguageprocessing #clustering