It is true that $(0,0,0)$ is Lyapunov stable.
The general idea has been suggested in other solutions, here are the truly gory details.
Let $p(t) = (x(t), y(t), z(t))$.
We need to establish that a unique solution exists for all time. Let $r>0$ and let $K_r = \overline{B}(0,r)$. Since the system is smooth, it is Lipschitz on any bounded set.
Let $p(t_0) \in B(0,\frac{1}{2}r)$ be an initial condition at time $t_0$. By the existence and uniqueness theorem, a solution exists in $K_r$ for some small time interval $[t_0,t_1]$. As above, we notice that $x(t)^2+2 y(t)^2=x_0^2+ y_0^2$, and hence $(x(t),y(t))$ remains bounded. Since $\dot{z} = x y$, we have $|z(t)-z(t_0)| \leq \frac{1}{2}(x_0^2+ y_0^2)(t-t_0)$. Consequently, we see that for any given $t_1$ we can choose $r$ large enough so that $p(t) \in K_r$ for all $t\in [t_0,t_1]$. It follows that the solution exists and is unique for all $t \geq 0$.
Now we will establish that $t \mapsto p(t)$ is periodic. We know that $(x(t),y(t))$ lies on the ellipse $x^2+2y^2 = V_0$, where $V_0 = x_0^2+2 y_0^2$. First we will show that $(x(t),y(t))$ traverses the entire ellipse without $z(t)$ growing 'too much'. Then we will show that $z(t)$ is periodic as well.
We know that $|z(t)| \leq |z(t_0)|+\frac{1}{2}V_0 (t-t_0)$. Hence if $|z(t_0)| < \frac{1}{4}$ and $(t-t_0) < \frac{1}{2 V_0}$, then $|z(t)| < \frac{1}{2}$. In particular, for any $T>0$, if $V_0$ and $|z(t_0)|$ are 'small enough', then $|z(t)| < \frac{1}{2}$ for any $t \in [t_0,t_0+T]$.
Since $(x(t),y(t))$ lie on the ellipse, we can write $x(t) = \sqrt{V_0} \cos \theta(t)$, $y(t) = \sqrt{\frac{v_0}{2}} \sin \theta(t)$ for some $C^1$ function $\theta$. Differentiating gives
\begin{eqnarray}
\dot{x}(t) &=& -\sqrt{V_0}\sin \theta(t) \dot{\theta}(t) &=& -2 \sqrt{\frac{v_0}{2}} \sin \theta(t) (1-z) \\
\dot{y}(t) &=& \sqrt{\frac{v_0}{2}} \cos \theta(t) \dot{\theta}(t) &=& \sqrt{V_0} \cos \theta(t) (1-z)
\end{eqnarray}
from which it follows that $\dot{\theta}(t) = \sqrt{2} (1-z)$. Now choose $T = 4 \sqrt{2} \pi$, which gives a neighborhood $U$ of $(0,0,0)$ such that if $p(t_0) \in U$, then $|z(t)| < \frac{1}{2}$ for any $t \in [t_0, t_0+T]$. Then we have $\dot{\theta}(t) \geq \frac{1}{\sqrt{2}}$, and hence $\theta(t_0+T)-\theta(t_0) \geq \frac{1}{\sqrt{2}} T = 4\pi$ (I choose $4 \pi$ so I can find a point where the positive $x$-axis is crossed and still have $2 \pi$ 'left to go'). Hence $(x(t),y(t))$ traverses the entire ellipse in some $[t_0,t_0+\delta] \subset [t_0, t_0+T]$. (It does not yet follow that $(x(t),y(t))$ is periodic.)
Let $\tau_0 \geq t_0$ be the first time at which $(x(t),y(t))$ crosses the positive $x$-axis, let $\tau_1 > \tau_0$ be the first time at which $(x(t),y(t))$ crosses the positive $y$-axis, and let $\Delta = \tau_1 -\tau_0$. For $t \in [\tau_0+\Delta, \tau_0+2\Delta]$, define $\tilde{x}(t) = -x(2\tau_0+2 \Delta-t)$, $\tilde{y}(t) = y(2\tau_0+2 \Delta-t)$ and $\tilde{z}(t) = z(2\tau_0+2 \Delta-t)$. It is straightforward to verify that $\tilde{p} = (\tilde{x},\tilde{y},\tilde{z})$ satisfies the differential equation with initial condition $\tilde{p}(\tau_0+\Delta)=p(\tau_0+\Delta)$.
By uniqueness, we have $p(t) = \tilde{p}(t) $ for $t \in [\tau_0+\Delta, \tau_0+2\Delta]$, and hence $(x(\tau_0+2\Delta), y(\tau_0+2\Delta),z(\tau_0+2\Delta))=(-x(\tau_0), y(\tau_0),z(\tau_0))$ (note that $y(\tau_0) = 0$ by choice of $\tau_0$).
Now we repeat the process for the remaining semi-ellipse. For $t \in [\tau_0+2\Delta, \tau_0+4\Delta]$, define $\tilde{x}(t) = x(2\tau_0+4 \Delta-t)$, $\tilde{y}(t) = -y(2\tau_0+4 \Delta-t)$ and $\tilde{z}(t) = z(2\tau_0+4 \Delta-t)$. Repeating the above procedure, we obtain $p(t) = \tilde{p}(t) $ for $t \in [\tau_0+2\Delta, \tau_0+4\Delta]$, and hence $(x(\tau_0+4\Delta), y(\tau_0+4\Delta),z(\tau_0+4\Delta))=(x(\tau_0), -y(\tau_0),z(\tau_0)) = p(t_0)$ (note that $y(\tau_0) = 0$ by choice of $\tau_0$).
It follows by uniqueness that $p(t+4 \Delta) = p(t)$ for all $t\geq t_0$. Furthermore, $|z(t)| \leq |z(t_0)| +\frac{1}{2}V_0 4 \Delta$ for all $t\geq t_0$.
Hence for all $\epsilon > 0$, there is a neighborhood $U$ of $(0,0,0)$ such that if $p(t_0) \in U$, then $\|p(t)\| <\epsilon$ for all $t \geq t_0$.
Best Answer
I think you need to read a bit further in the tutorial to clarify this point. At the beginning of the tutorial, the author suggests that $f(x)$ is a candidate for a Lyapunov functional. Note that the function $f(x)$ is not unique - any $f(x)+c$ would also be a potential for the vector field $\nabla f(x)$. Therefore, you can just pick $f_2 (x)=f(x)-f(0)$ as your potential function, and it will automatically satisfy $f_2 (0)=0$. This is to say that condition 1 can basically be automatically satisfied. The author addresses your point about condition 2 in Theorem 2.3, in which they state that $f_2 (x)$ will be a Lyapunov function when $0$ is a strict local minimum of $f$. This will satisfy your criteria 2, at least locally. I hope this helps!