An activation function is a function used in artificial neural networks which outputs a small value for small inputs, and a larger value if its inputs exceed a threshold.
If the inputs are large enough, the activation function "fires", otherwise it does nothing.
In other words, an activation function is like a gate that checks that an incoming value is greater than a critical number.
I compared Sigmoid and Softmax activation functions, then demonstrated the differences in Python.
You are welcome to provide your comments and subscribe to my YouTube channel.
The Python code is uploaded into https://github.com/AIMLModeling/Softmax