An activation function is a function used by an artificial neuron (represented as a node in a graph) of an ANN (i.e. the weighted directed graph) which produces some output given one or more inputs to the neuron. There are many different activation functions.
In this answer, an activation of a neuron is defined as a function of its input: specifically, it's a transformation of this input, which is performed via an "activation function".
Finally, the author of the same answer also says that the activation of a neuron can be interpreted as an internal representation of the input.
Is there any difference between an activation and an action function of an artificial neuron in an ANN? If yes, which one(s)?
I suppose these are just two terms that, in practice, refer to the same thing, i.e. a mathematical function, but, for some reason, in theory, some people use these two terms. If I am correct, what's this reason?
Best Answer
The activation of a unit $U$ (a.k.a. 'artificial neuron') is the value that $U$ takes in response to its input. $U$ passes this value to downstream units, who receive it as input. So, one could also refer to the activation as the output of $U$. The activation function is a function used to compute the activation.
For example, consider a unit that receives $d$ inputs, stored in vector $x \in \mathbb{R}^d$. It has weight vector $w$ and bias $b$. The activation is given by:
$$f(x) = \sigma(w \cdot x + b)$$
The activation function is $\sigma: \mathbb{R} \rightarrow \mathbb{R}$. Notice that the unit itself takes a $d$ dimensional vector of reals as input and returns a single real number as output. But, the activation function maps a single real number ($w \cdot x + b$) to a single real number (the activation). Common choices of activation function include sigmoidal functions (e.g. $\tanh$), rectified linear functions, and the identity function (a.k.a. linear activation function).