So basically I wanted to figure out why the differential of the following expression is what it is:
\begin{align} S &= e'e \\ &= (y-W\beta)'(y-W\beta) \\ &= \underbrace{y'}_{1\times T} \ \underbrace{y}_{T\times 1} -\underbrace{y'}_{1\times T} \ \underbrace{W\beta}_{(T\times2)(2\times1)} -\underbrace{\beta'W'y}_{(1\times2)(2\times T)(T\times1)} +\underbrace{\beta'W'W\beta}_{1\times1},
\end{align}
$$S =y'y -2\beta'W'y +\beta'W'W\beta.$$
$$\frac{\partial}{\partial \beta} = -2W'y + W'W\beta$$
Basically, I don't understand why the differential is not $$-2y'W$$ + the other term which I don't even know how to derive.
For how I got my result, I followed this guide. https://atmos.washington.edu/~dennis/MatrixCalculus.pdf
Thanks in advance.
Best Answer
It's simpler to differentiate before expanding the $e$ vector. $$\eqalign{ S &= e^Te \cr dS &= 2e^Tde = 2e^T(W\,d\beta) = (2W^Te)^Td\beta \cr \frac{\partial S}{\partial\beta} &= 2W^Te = 2W^T(W\beta - y) \cr }$$