Derivative of a vector with respect to a scalar

derivativesmultivariable-calculus

I'm reading this paper: https://explained.ai/matrix-calculus/index.html#sec4.3 , under the section- Derivatives Involving Scalar Expansion the following is written, and at the end of this section is the equation I've written here:

"$\vec{f}(\vec{x})=\vec{x}$ and $\vec{g}(z)= \vec{1} z$. (The notation represents a vector of ones of appropriate length.) $z$ is any scalar that doesn't depend on $x$, which is useful because then $\dfrac{\partial{z}}{\partial{x_i}}=0$ for any $x_i$ and that will simplify our partial derivative computations. (It's okay to think of variable z as a constant for our discussion here)."

$$\dfrac{\partial(f_i(x_i)\circ g_i(z))}{\partial z}=x_i\dfrac{\partial z}{\partial z}+z\dfrac{\partial x_i}{\partial z}=x_i+?$$

What's the $?$, I think it's undefined as derivative with respect to a constant is undefined, but the paper says it's $0$, how so?

Any help is highly appreciated.

Best Answer

Well, if you consider $x_i\frac{\partial z}{\partial z}+z\frac{\partial x_i}{\partial z}$, we can immediately see that $\frac{\partial z}{\partial z}=1$, since intuitively, z changes at the same rate as itself. Symbolically, we could write that $\frac{\partial x_k}{\partial x_l}=\delta^k_l$, where $\delta^k_l=1$ when $k=l$ and $0$ otherwise.

Now, it was explicitly stated that $z$ does not depend on $x$, therefore $\frac{\partial z}{\partial x_i}=0$. As $z$ does not depend on $x_i$, the converse is also true; namely, $x_i$ does not depend on $z$, since else you could express $z$ in terms of $x_i$, which would contradict the initial assumption. Hence, $\frac{\partial x_i}{\partial z}=0$. This implies that regardless of the value of $z$, the second term is zero.

Related Question