Knowledge Distillation with TAs
This video presents a paper exploring the effectiveness of introducing intermediate teaching networks (Teacher Assistants) to the knowledge distillation pipeline! Thanks for watching, please subscribe!
Improved Knowledge Distillation via Teacher Assistant:
https://arxiv.org/pdf/1902.03393.pdf
On the Efficacy of Knowledge Distillation:
https://arxiv.org/abs/1910.01348
EfficientNet:
https://arxiv.org/abs/1905.11946
Self-Training with Noisy Student:
https://arxiv.org/abs/1911.04252