Let $M$ be a smooth manifold.
Let $p\in M$. A tangent vector at $p$ is a an equivalence class $[\gamma]$ of smooth curves $\gamma : (-\epsilon,\epsilon)\rightarrow M$, with $\gamma(0) = p$, where the equivalence relation is as follows: $$\gamma_1\sim\gamma_2 \iff \exists \text{ chart } (U,\phi) \text{ such that } (\phi\circ\gamma_1)'(0) = (\phi\circ\gamma_2)'(0)$$
One can now define the tangent space $T_pM$ as the set of all tangent vectors. And define the derivative as follows:
Let $f:M\rightarrow N$ be a differentiable map between manifolds, $p\in M$. The derivative of $f$ at $p$ is $$(f_*)_p:T_pM\rightarrow T_pN: [\gamma]\mapsto[f\circ\gamma] $$
I understand that the derivative at $f$ just changes the tangent vectors at $M$ to tangent vectors at $N$ such that it has nice properties. But I'm having a hard time understanding what this means practically of how to interpret these tangent vectors.
The exercise I am trying to solve defines $f:S^2\rightarrow \mathbb{R}:(x,y,z)\mapsto z^2$ and asks for which $p$ $(f_*)_p = 0$.
I have proven that if $F:M\rightarrow \mathbb{R}$ and $N$ a submanifold, that if $f = F\mid_N:N\rightarrow \mathbb{R}$ that for all $p\in N$: $(f_*)_p = 0\iff T_pN\subset ker(F_*)_p$.
It seems like this can be useful in some way, but then again I think I need to calculate $(F_*)_p$ which I don't know how to do.
Best Answer
The theorem you have indeed is useful, provided you know what the tangent space to $N$ looks like/ if you can easily figure it out, and provided you know what $F,M$ should be. In your particular case, it should be natural to choose $M= \Bbb{R}^3$, equipped with the maximal atlas containing the identity chart $(\Bbb{R}^3, id)$. Then, of course, we take $N = S^2$, and lastly, it should be reasonable to define $F: \Bbb{R}^3 \to \Bbb{R}$ by \begin{align} F(x,y,z) = z^2 \end{align}
Then, clearly, $f = F|_N$.
Now, let's side-track slightly and see how to think of tangent vectors as actual elements of your model space, so that we can see how to apply this to your question. Let $X$ be a smooth manifold (I avoid $M$ because I already used it above) of dimension $n$. Let $p \in X$, and let $(U, \alpha)$ be any chart of $X$ containing the point $p$ (i.e $p \in U$). Then, one can construct an isomorphism of $T_p(X)$ onto $\Bbb{R}^n$ as follows: define the map $\Phi_{\alpha, p}: T_pX \to \Bbb{R}^n$ by \begin{align} \Phi_{\alpha, p}\left( [c] \right) &:= (\alpha \circ c)'(0) \end{align} By definition of the equivalence relation, this map is well defined (in fact it is pretty much using this map that you use to get a vector space structure on $T_pX$).
Now, if $g: X \to Y$ is a smooth map between smooth manifolds, and $(U, \alpha)$ is a chart about a point $p \in X$, and $(V, \beta)$ is a chart for $Y$ about the point $g(p)$, then we have the following commutative diagram:
$\require{AMScd}$ \begin{CD} T_pX @>{g_{*p}}>> T_{g(p)}Y \\ @V{\Phi_{\alpha,p}}VV @VV{\Phi_{\beta, g(p)}}V \\ \Bbb{R}^n @>>{D(\beta \circ g \circ \alpha^{-1})_{\alpha(p)}}> \Bbb{R}^m \end{CD} Which I'll leave to you to verify that it is actually commutative. In words what this means is that to compute $g_{*,p}$, you can choose charts on the domain and the target, and instead consider the (familiar euclidean type) derivative of the chart-representative map $\beta \circ g \circ \alpha^{-1}$.
Now, a special case of interest is the following: we have $X$ as a submanifold of some $\Bbb{R}^l$, and we're given a point $p \in X$. How do we think of $T_pX$ in an intuitive way? Well, $p$ is an element of $X$ and hence $\Bbb{R}^l$, so it (trivially) lies in the identity chart $(\Bbb{R}^l, id)$. Thus, rather than thinking of the tangent space as $T_pX$, which set-theoretically consists of equivalence classes of curves (which is rather abstract and tough to compute with), consider instead the $\dim X$-dimensional subspace $\Phi_{id,p}\left( T_pX \right) \subset \Bbb{R}^l$. This is precisely the intuitive picture of tangent space you would have.
For example, if $X= S^2$ considered as a submanifold of $\Bbb{R}^3$, then for each $p \in X= S^2$, $\Phi_{id,p}(T_pS^2)$ will be the (translated) tangent plane $\{\xi \in \Bbb{R}^3: \langle \xi, p\rangle = 0 \}$ (which is precisely the usual intuitive picture you might have). To prove this in a rigorous manner, it will be much easier if you know that $S^2$ can be written as a level set, say $h^{-1}(\{1\})$, where $h: \Bbb{R}^3 \to \Bbb{R}$ is defined by $h(x,y,z) = x^2 + y^2 + z^2$, and that $T_pS^2 = \ker h_{*p}$, so that (by the above commutative diagram, and basic linear algebra), \begin{align} \Phi_{\text{id}_{\Bbb{R}^3}, p}(T_pS^2) = \ker D(\text{id}_{\Bbb{R}} \circ h\circ \text{id}_{\Bbb{R}^3}^{-1})_{\text{id}_{\Bbb{R}^3}(p)} = \ker(Dh_p) \end{align} where $Dh_p : \Bbb{R}^3 \to \Bbb{R}$ is the usual derivative.
So, now back to your question. You seek all $p \in S^2$ such that $f_{*,p} = 0$, or equivalently, by your theorem, those $p \in S^2$ such that $T_pS^2 \subset \ker F_{*,p}$. Or equivalently, those $p \in S^2$ such that \begin{align} \Phi_{\text{id}_{\Bbb{R}^3}, p}(T_pS^2) \subset \Phi_{\text{id}_{\Bbb{R}^3}, p} \left( \ker F_{*,p} \right) = \ker DF_p \end{align} So, to answer your question, you just have to compute $DF_{p}$ for all $p \in S^2$ (in the usual calculus sense), compute the kernel of this map, and then see whether the translated tangent plane to the sphere lies inside the kernel.
I believe that this final computational part isn't difficult so I'll leave this all to you. I felt it is more important to see the logic behind what kind of computation needs to be performed.
Remark:
In your particular question, you've been aided by the fact that the theorem you stated gives a nice short proof (after a bit of practice, the reasoning I explained above in gory detail will become natural, so you'll be able to directly jump to my paragraph above... so this really is a short solution). However, suppose that you didn't know about that theorem. Then how would you go about finding the set of $p$ where $f_{*p} = 0$?
Well, the answer is very simple and straight forward (perhaps algebraically more tedious if you don't remember the charts). The sphere $S^2$ is a manifold, and as such, it has charts. The sphere is so nice that it can be covered by $2$-charts, (if you use stereographic projection).
Consider the stereographic projection from the north-pole: let $U_N = S^2 \setminus \{(0,0,1)\}$, and $\sigma_N : U_N \to \Bbb{R}^2$ \begin{align} \sigma_N(x,y,z) = \left( \dfrac{x}{1-z}, \dfrac{y}{1-z} \right) \end{align} Its inverse is $\sigma_N^{-1}: \Bbb{R}^2 \to U_N$ \begin{align} \sigma_N^{-1}(\xi,\eta) = \left( \dfrac{2\xi}{\xi^2 + \eta^2 +1}, \dfrac{2\eta}{\xi^2 + \eta^2 +1}, \dfrac{\xi^2 + \eta^2 - 1}{\xi^2 + \eta^2 +1} \right) \end{align}
This chart covers the whole $S^2$ except the north-pole $(0,0,1)$. Now, it should be easy enough to verify that for $p \in U_N$, $f_{*,p} = 0$ if and only if $D(f \circ \sigma_N^{-1})_{\sigma_N(p)} = 0$. Or said differently, $f_{*, \sigma_N^{-1}(\xi,\eta)} = 0$ if and only if $D(f \circ \sigma_N^{-1})_{(\xi,\eta)} = 0$. But \begin{align} f \circ \sigma_N^{-1}(\xi, \eta) = \left( \dfrac{\xi^2 + \eta^2 - 1}{\xi^2 + \eta^2 +1} \right)^2 \end{align} So, it should be easy to compute the standard derivative, and find where it vanishes. Then lastly, you just have to see if $f_{*, (0,0,1)} = 0$. To do this, you have to choose a chart which covers the north pole; you could use the stereographic projection from the south pole, or you could instead use the much simpler "graph chart" given by $V_{z,+} = \left\{(x,y,z) \in S^2| \, z>0 \right\}$ and $\psi_{z,+} : V_{z,+} \to \{(x,y)|\, x^2 + y^2 < 1\}$ given by \begin{align} \psi_{z,+}(x,y,z) = (x,y) \end{align} Note that I restricted the domain and target so that this is invertible, with inverse \begin{align} \psi_{z,+}^{-1}(x,y) = (x,y,\sqrt{1-x^2-y^2}) \end{align} (I chose postive square root because of the definition of $V_{z,+}$). Hence, in this case, \begin{align} (f \circ \psi_{z,+}^{-1})(x,y) = 1-x^2-y^2 \end{align} So, $f_{*, (0,0,1)} = 0$ if and only if $D(f \circ \psi_{z,+}^{-1})_{\psi_{z,+}(0,0,1)} = D(f \circ \psi_{z,+}^{-1})_{(0,0)} = 0$. Again, it should be easy enough to verify whether or not this condition is satisfied.
To recap: if you did not know that theorem, you just find an atlas for the manifold (and for computational purposes, find one with the fewest/simplest charts). Then, any property you wish to investigate about the push-forward $f_{*p}$ can be phrased equivalently in terms of the derivatives of the chart-representative maps, and solve the question in the chart (this is useful in general too).
For instance, if you hada slightly tougher question, say you're given some map $g: S^3 \to \Bbb{R}^4$, and you were asked to find all points where $g_{*p}$ had full rank, then I think a coordinate approach would be very mechanical and straight-forward. (As much as possible, it is a good idea to avoid charts, but it is also good to get used to them, because sometimes, they can provide a much quicker solution.)