This question is about derivatives. It is okay if you don't know backpropagation.
I was following this guide to understand backpropagation algorithm
(equation 36, 37 and 38).
Here is a simplified form of the problem. Let
$$z_1 = w_{11}*x_1 + w_{21}*x_2 + w_{31}*x_3$$
$$z_2 = w_{12}*x_1 + w_{22}*x_2 + w_{32}*x_3$$
And,
$$a_1 = F(z_1)$$
$$a_2 = F(z_2)$$
We also have a function $C(a_1,a_2)$
Now they applied multivariable chain rule to find,
$$\frac{\partial C}{\partial z_1} = \frac{\partial C}{\partial a_1}\frac{\partial a_1}{\partial z_1} + \frac{\partial C}{\partial a_2}\frac{\partial a_2}{\partial z_1}$$
Then they stated that $$\frac{\partial a_2}{\partial z_1} = 0 $$ I don't understand why is this term equal to 0?
According to what i learnt from this answer, i think it should be
$$\frac{\partial a_2}{\partial z_1} = F'(z_2)\frac{\partial z_2}{\partial z_1} = F'(z_2)(\frac{\partial z_2}{\partial x_1}\frac{\partial x_1}{\partial z_1}+\frac{\partial z_2}{\partial x_2}\frac{\partial x_2}{\partial z_1}+\frac{\partial z_2}{\partial x_3}\frac{\partial x_3}{\partial z_1})$$
$$\frac{\partial a_2}{\partial z_1} = F'(z_2)(\frac{w_{12}}{w_{11}} + \frac{w_{22}}{w_{21}} + \frac{w_{32}}{w_{31}})$$
Please tell me where i am wrong
Best Answer
Now I think I understand. Based on the tutorial you are referencing, the $x_i$'s are not your variables, they are your data. Instead, the variables $z_1$ and $z_2$ are functions of the the weights $w_{11}, w_{12}, \ldots$.
Here is an updated diagram that may help. You are concerned with the output of the node containing $z_2$. Your function $F$ is only being applied to the output from $z_2$. There is no connection that takes you directly from $z_1$ to $z_2$. That is, the derivative is 0.
Note that this is a picture of the variable dependence not your neural network. In the network diagram, the $w$'s are the edges.
Your equations now look like
$$ \begin{align} a_2 &= F(z_2)\\ z_2 &= f(w_{12},w_{22},w_{32})\\ z_1 &= f(w_{11},w_{21},w_{31}) \end{align} $$
So it is clear that
$$ \frac{\partial a_2}{\partial z_1} = 0 $$