To answer, I assume $F(x)$ and $G(x)$ are continuous in $I \in \mathbb{R}, I = [a,b]$. For simplicity, let $F(x)=f, G(x) = g, H(x) = h = f-g$. By symmetry I can assume $f \ge g$. Now consider the function $h(x)$:
- if $\exists x | h(x) = 0$, it means that there is a point $x$ such that $f(x) = g(x) \Rightarrow ||f(x) - g(x)|| = 0$ and so the minimum shortest distance between $f$ and $g$ is 0
- else, find the minimum value of $h$ (by letting $h'=0$): let's call the minimum $m$. Let $P$ be the point found intersecating $f$ and the normal (i.e the tangent to the tangent to the curve) to $g$, and $N$ vice-versa (please, take a look at the image below). You need to fix a point $a$ on $f | P \le a \le f(m)$ and a point $b$ on $g| g(m) \le b \le N$, then the shortest distance is $\min[||a-b||]$.
There doesn't seem to be a nice closed-form solution to this, but the following might be helpful..
For an ellipse $\frac {x^2}{a^2}+\frac {y^2}{a^2}=1$, the equation of the normal at point $P(a \cos\theta, b\sin\theta)$ on the ellipse is
$$\frac {a\sin\theta}{b\cos\theta}x-y=a\left(\frac{a^2-b^2}{ab}\right)\sin\theta\tag{1}$$
A hyperbola which just touches or does not intersect at all with the ellipse above has the equation $xy=\frac {mab}2$ where $m\ge 1$. The equation of the normal at point $Q (v,\frac {mab}{2v})$ on the hyperbola is
$$\frac {2v^2}{mab}x-y=v\left(\frac {2v^2}{mab}-\frac {mab}{2v^2}\right)\tag{2}$$
The minimum distance between the ellipse and hyperbola is the distance $PQ$ when $(1)=(2)$, i.e. both normals are the same line.
As the coefficients of $y$ are the same in both $(1),(2)$, equating coefficients of $x$ in $(1),(2)$ gives
$$v=a\sqrt \frac{m\tan\theta}{2}\tag{3}$$
This relationship ensures that the tangents and normals at $P,Q$ are parallel to each other respectively (but the normals are not necessarily the same line).
Putting $(3)$ in $(2)$ gives
$$\left(\frac ab \tan\theta\right)x-y=a\sqrt{\frac{m\tan\theta} 2}\left(\frac ab\tan\theta-\frac ba\cot\theta\right)\tag{4}$$
To ensure that both normals are the same line, we need to equate RHS of $(1),(4)$. This gives
$$\left(\frac{a^2-b^2}{ab}\right)\sin\theta=\sqrt{\frac{m\tan\theta}2}\left(\frac ab \tan\theta-\frac ba\cot\theta\right)$$
which is equivalent to
$$(a^2-b^2)\sin\theta=\sqrt{\frac{m\tan\theta}2}\left(\frac{a^2\sin^2\theta-b^2\cos^2\theta}{\sin\theta\cos\theta}\right)\tag{4}$$
Solve numerically $\theta$ in $(4)$, find corresponding value $v$ in $(3)$, then calculate $PQ$. This should give the minimum distance between the ellipse and hyperbola.
See desmos implementation here.
In the trivial case where $a=b$ (i.e. ellipse is a circle), then $\theta=\frac \pi 4$ and $v=a\sqrt{\frac m2}$ . This gives $P=\left(\frac a{\sqrt{2}}, \frac a{\sqrt{2}}\right)$ and $Q=\left(a\sqrt{\frac m2}, a\sqrt{\frac m2}\right)$ and the distance $PQ=a(\sqrt m-1)$.
Best Answer
We can assign parametric coordinates to the curves. Let them be $\left(a^2+\frac12,a\right)$ and $\left(b,b^2+\frac12\right)$
Distance between any two points will be $$\sqrt{\left(a^2+\frac12-b\right)^2+\left(b^2+\frac12-a\right)^2}$$ Now using some inequalities we can say (it's the quadratic mean inequality) $$\sqrt{\left(a^2+\frac12-b\right)^2+\left(b^2+\frac12-a\right)^2}\ge\frac{a^2+b^2-a-b+1}{\sqrt{2}}=\frac{\left(a-\frac12\right)^2+\left(b-\frac12\right)^2+\frac12}{\sqrt2}$$ which has a minimum value equal to $\frac{1}{2\sqrt2}$ Hence the minimum distance between the two curves is $$\boxed{\frac{1}{2\sqrt2}}$$