General AI | Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator
If you enjoyed this video, feel free to LIKE and SUBSCRIBE; also, you can click the 🔔 for notifications!
If you would like to support the channel, please join the membership:
/ aipursuit
Subscribe to the channel:
https://www.youtube.com/c/AIPursuit?s...
The video is reposted for educational purposes and encourages involvement in the field of research.
Source: https://slideslive.com/38953694/raobl...
0:00 Introduction
0:13 Discrete Data
1:03 Example: Categorical Variational Autoencoder (VAE)
2:38 Taxonomy of Gradient Estimators
4:07 Review: Gumbel-Softmax (GS)
6:52 Properties of Gumbel-Rao Monte Carlo
7:21 Zooming out: Trading off computation and variance
8:32 Extensions to other structured variables
8:55 Experiments
9:26 Toy problem: Quadratic programming on the simplex
9:55 Variance improvements at different temperatures
10:46 Categorical VAE on MNIST
11:26 Negative log-likelihood lower bounds on MNIST
12:10 Variance and MSE for gradient estimation
12:56 Conclusion