Visualize Back Propagation: (5) Momentum
In this video we look at the effect of momentum on the path we traverse in the weight-space. We also look at an updated version of momentum put forth originally by Nesterov and researched recently my Ilya Sutskever which he found empirically to be better.