You only need to consider the case $\mathfrak{h}_{s}^\ast(A) \lt \infty$, but you need to be a bit careful in choosing the outer approximations since swapping $\inf$ and $\sup$ certainly isn't allowed without some thinking. If you knew that you can always take the same set $E$ in the $\inf$ (which I will show in \eqref{eqn:ast} below) then you'd be essentially done, but it is impossible to say if you got so far or not.
I'm going to ignore $\alpha_s$ since it only is a scalar factor that doesn't play a rôle in the argument and the formulas will already get cumbersome enough without it.
The following argument is essentially the one from Fremlin's measure theory, Volume 4II, 471D, page 102. $\DeclareMathOperator{\diam}{diam}$ $\newcommand{\hsd}[1]{\mathfrak{h}_{s,#1}^\ast}$
Set $\delta_n = 2^{-n}$ and choose $(A_{i}^n)_{i \in \mathbb{N}}$ of diameter $\leq 2^{-n}$ such that $A \subset \bigcup_{i=1}^\infty A_{i}^n$ and that
\begin{equation}\tag{$1$}\label{eqn:eq1}
\sum_{i=1}^\infty (\diam{A_{i}^n})^s \leq \hsd{2^{-n}}(A) +2^{-n},
\end{equation}
which is possible because of the definition of $\hsd{2^{-n}}$ as infimum.
Observe that there's no reason for the $A_{i}^n$ to be $\hsd{2^{-n}}$-measurable, let alone Borel, so we would like to “blow them up” slightly, so as to get open sets still approximating the $\hsd{2^{-n}}$-measure of $A$ well. The problem is that by doing so we will lose the diameter condition which appears in the definition of $\hsd{2^{-n}}$, but this isn't a serious problem: we can simply choose a larger $n$ and work with the sets we obtain from there. Here are the gory details:
Choose $0 \lt r_{i}^n \lt 2^{-n}$ so small that
$$
(\diam{(A_{i}^n)} + 2r_{i}^n)^s \leq (\diam{A_{i}^n})^s + 2^{-n-i}.
$$
Now put $U_{i}^n = \{x \in X\,:\,d(x,A_{i}^n) \lt r_{i}^n\}$ and note that $U_{i}^n \supset A_{i}^n$ is an open set whose diameter satisfies
\begin{equation}\tag{$2$}\label{eqn:eq2}
(\diam{U_{i}^n})^s \leq (\diam{A_{i}^n})^s + 2^{-n-i}.
\end{equation}
Let
$$
B = \bigcap_{n=1}^\infty \bigcup_{i=1}^\infty U_{i}^n
$$
and observe that $B$ is a $G_{\delta}$-set (countable intersection of open sets) containing $A$.
Given any $\delta \gt 0$, we can find $N$ such that $3 \cdot 2^{-N} \leq \delta$ so that for all $n \geq N$ we have $\diam{U_{i}^n} \leq \delta$. As $B \subset \bigcup_{i = 1}^\infty U_{i}^n$ we see from \eqref{eqn:eq1} and \eqref{eqn:eq2} that for $n \geq N$
$$
\hsd{\delta}(B) \leq \sum_{i=1}^\infty (\diam{U_{i}^n})^s
\leq
\sum_{i=1}^\infty [(\diam{A_{i}^n})^s + 2^{-n-i}] \leq \hsd{2^{-n}}{(A)} + 2 \cdot 2^{-n}
$$
Since $\hsd{2^{-n}}(A) \leq \mathfrak{h}_{s}^\ast(A)$ we get for $n \geq N$
$$
\hsd{\delta}(B) \leq
\mathfrak{h}_{s}^\ast(A) + 2^{-n}
$$
and letting $n \to \infty$ this gives
\begin{equation}\tag{$\ast$}\label{eqn:ast}
\hsd{\delta}(B) \leq \mathfrak{h}_{s}^\ast(A)
\end{equation}
for every $\delta \gt 0$. [Note: this is stronger than your condition involving $\inf$ since we can specify the set $E = B$ and thus avoid the infimum]
Taking the $\sup$ over all $\delta$ in \eqref{eqn:ast} this yields
$$
\mathfrak{h}_{s}^\ast(B) \leq \mathfrak{h}_{s}^\ast(A)
$$
and since $B \supset A$ and $B$ is $\mathfrak{h}_{s}$-measurable (being Borel) we can finally conclude
that
$$
\mathfrak{h}_{s}(B) = \mathfrak{h}_{s}^\ast(A),
$$
as desired.
Of course there is a general theorem saying that for a smooth arc $\gamma$ the one-dimensional Hausdorff measure of this arc is equal to its length computed via an integral.
In the case at hand the length $L(\gamma)$ of the arc
$$\gamma:\quad x\mapsto(x,x^2)\qquad(0\leq x\leq 2)$$
is defined as
$$L(\gamma):=\sup_{\cal P}\sum_{k=1}^{N_{\cal P}}\sqrt{(x_k-x_{k-1})^2+(x_k^2-x_{k-1}^2)^2}\ ,\tag{1}$$
where the $\sup$ is taken over all partitions ${\cal P}$ of the $x$-interval $[0,2]$. It is proven in calculus 102 that this $L(\gamma)$ is equal to the integral you have calculated.
On the other hand arguments similar to those used in the proof of $(1)$, but "the other way around", show that $H^1(\gamma)$, as defined in the question, is equal to the same integral.
Best Answer
It's enough to do a projection from $\partial(F)$ to $\partial E$.
This is a Lipshitz function with Lipshitz constant 1 and so by the properties of the Hausdorff measure we know that $H^s(f(E))\le L^s (H^s(E))$ and so we are done