Sequence to Sequence Learning with Neural Networks: The Original Seq2Seq Paper
A deep dive into the seminal 2014 paper by Sutskever, Vinyals, and Le that introduced the encoder-decoder LSTM architecture for general sequence-to-sequence learning, achieving a BLEU score of 34.8 on WMT'14 English-to-French translation and surpassing the phrase-based SMT baseline.
In this video, we walk through the core idea of using one LSTM to encode an input sequence into a fixed-dimensional vector and a second LSTM to decode the target sequence, the surprising trick of reversing the source sentence to introduce short-term dependencies, and the experimental results on translation and SMT rescoring. We also explore the learned sentence representations and discuss why this architecture became the foundation for modern neural machine translation and, eventually, the Transformer era.
The video concludes with a Q&A section addressing common questions about the model's design choices, optimization tricks, and lasting impact on NLP.
https://arxiv.org/abs/1409.3215