Some Math in AI
We look briefly at 7 math topics which appear in modern transformer networks. The universal approximation theorem is of analytic nature, data fitting is used to reduced data, activation functions appear in single variable calculus, embeddings resemble parametrizations and produce taxonomies, graadient methods allow using backgracking to tune the weights, the soft max distribution is not only important in physics as it is the Gibbs distribution, it also minimizes free energy.