Deep Learning: 03: What is a Neural Network? (Math Explained)
Two major operations of a neural network:
1. Linear transformations; i.e., we multiply inputs by weights matrix and add a “bias” vector
2. Point-wise non-linearities; i.e., we pass the output to a non-linear function like ReLU