KL Divergence - How to tell how different two distributions are
Correction (10:26). The probabilities are wrong. The correct ones are here:
For Die 1: 0.4^4 * 0.2^2 * 0.1^1 * 0.1^1 * 0.2^2
For Die 2: 0.4^4 * 0.1^2 * 0.2^1 * 0.2^1 * 0.1^2
For Die 3: 0.1^4 * 0.2^2 * 0.4^1 * 0.2^1 * 0.1^2
Kullback Leibler (KL) divergence is a way to measure how far apart two distributions are.
In this video, we learn KL-divergence in a simple way, using a probability game with dice.
Shannon entropy and information gain: • Shannon Entropy and Information Gain
Grokking Machine Learning book: www.manning.com/books/grokking-machine-learning
40% discount code: serranoyt