Definition of logistic sigmoid function is:
$$σ(t)=\frac{1}{1 + e^{-t}}$$
Differentiating this is very simple, It just requires simplification with exponent rule and differentiation with quotient rule.
$$\frac{d}{dx}\frac{1}{1 + e^{-t}}=\frac{d}{dx}\frac{e^{t}}{1 + e^{t}}=\frac{e^{t}(1 + e^{t}) -e^{2t}}{(1 + e^{t})^{2}}=\frac{e^{t}}{(1 + e^{t})^{2}}$$
But what if instead of $t$, the input was weighted sum $\sum{w_ix_i}$ ?
The partial derivative with respect to $w_i$ somehow becomes $σ*1-σ*x_{i}$, this is how one person calculated it:
$$\frac{\partial}{w_{i}}\frac{1}{1 + e^{-\sum{w_ix_i}}}=\frac{-1}{(1+e^{-\sum{w_ix_i}})^{2}}*e^{-\sum{w_ix_i}}*(-x_i)$$
$$\frac{\partial}{w_{i}}\frac{1}{1 + e^{-\sum{w_ix_i}}}=\frac{1}{(1+e^{-\sum{w_ix_i}})}*\frac{e^{-\sum{w_ix_i}}}{{(1+e^{-\sum{w_ix_i}})}}*x_i$$
Now, considering that $1-σ$ is $\frac{e^{-\sum{w_ix_i}}}{{(1+e^{-\sum{w_ix_i}})}}$, we get:
$$\frac{\partial}{w_{i}}\frac{1}{1 + e^{-\sum{w_ix_i}}}=σ*1-σ*x_i$$
Is this derivative properly calculated? If so how exactly? $x_i$ is somehow outside of summation, is it due to it being treated as constant?
Best Answer
You have a composite function, $\sigma=1/(1-e^{-t})$ where $t=\sum w_i x_i$, so just use the chain rule: $$ \frac{\partial \sigma}{\partial w_i} = \frac{d \sigma}{d t} \cdot \frac{\partial t}{\partial w_i} = \sigma (1-\sigma) \cdot x_i . $$ (See here for a slick way of deriving $\frac{d \sigma}{d t}=\sigma (1-\sigma)$.)