TensorFlow London: Text classification with transformers in TensorFlow 2 by David Mráz
Speaker: David Mráz, Co-founder at Atheros.ai
Bio: David Mráz is a co-founder at atheros.ai, software architect and machine learning engineer. He was previously a researcher at the Department of Mathematics FCE CTU, but now his focus is solely on helping companies to build AI-driven products. His research interests are especially focused on modern approaches in natural language processing and understanding.
Title: Text classification with transformers in TensorFlow 2
Abstract: The transformer-based language models have been showing promising progress on a number of different natural language processing (NLP) benchmarks. Many model architectures such as BERT, GPT-2, XLNet or recent Turing-NLG pushed the state of the art results on various different NLP tasks. The ability to pre-train the language model on large datasets in an unsupervised way and then finetune on downstream tasks has been one of the biggest reasons for such breakthroughs. The combination of transfer learning methods with large-scale transformer language models is becoming a standard in modern NLP. In this talk, we will make the necessary theoretical introduction to transformer architecture and text classification problem. Then we will demonstrate the fine-tuning process of the pre-trained BERT model for text classification in TensorFlow 2.