Here's a reasonable method: translate everything such that the center of the ellipse is at the origin. Consider the intersection of the ellipse with major axis $2a$ and minor axis $2b$ with the polar equation
$$r=\frac{a b}{\sqrt{b^2\cos^2\theta+a^2\sin^2\theta}}$$
and the line $\tan\,\theta=\dfrac{y_2-y_1}{x_2-x_1}$. (When solving the last equation for $\theta$, you will want to use the two-argument arctangent that is implemented in most computing environments.) Once having computed the corresponding values of $r$ at $\theta$ and $\pi+\theta$, convert to rectangular coordinates and translate back to your initialal origin.
Let $a,b$ be the major and minor axes of the ellipse.
Let $x_0,y_0$ be the coordinates of the center on the ellipse.
Let $x_1,y_1$ be the coordinates of the one point on the ellipse.
First, we write the equation of the ellipse, those major axis is parallel to the $x$ axis of the coordinate system (which is at this point completely arbitrary).
$$\frac{(x-x_0)^2}{a^2}+\frac{(y-y_0)^2}{b^2}=1$$
Now we need the ellipse to go through the point $x_1,y_1$. This requires first that the distance between this point and the center is somewhere between $a$ and $b$:
$$b \leq \sqrt{(x_1-x_0)^2+(y_1-y_0)^2} \leq a$$
If it's indeed true, we need to find the four corresponding points on the ellipse on the same distance from the center.
$$d=\sqrt{(x_1-x_0)^2+(y_1-y_0)^2}$$
One of them is a green point on the picture below:
We need to find the green point. Let's call it $x_2,y_2$. Then we have two equations in two variables:
$$\frac{(x_2-x_0)^2}{a^2}+\frac{(y_2-y_0)^2}{b^2}=1$$
$$(x_2-x_0)^2+(y_2-y_0)^2=(x_1-x_0)^2+(y_1-y_0)^2$$
After you solve this simple system of equations and find $x_2,y_2$, there is one last step left: you need to rotate your ellipse.
First, find the angle between lines going from the center of the ellipse to $x_1,y_1$ and to $x_2,y_2$. You can do it by using vector dot product:
$$\cos \theta=\frac{(x_1-x_0)(x_2-x_0)+(y_1-y_0)(y_2-y_0)}{\sqrt{(x_1-x_0)^2+(y_1-y_0)^2}\sqrt{(x_2-x_0)^2+(y_2-y_0)^2}}$$
The rotated coordinates can be then found from the following system of equations:
$$x=X \cos \theta-Y \sin \theta \\ y=X \sin \theta+ Y \cos \theta$$
Note, that I inverted the problem - as if the ellipse is already rotated counter-clockwise, and we want to rotate it back. Meaning, we just substitute these formulas in the original equation:
$$\frac{(X \cos \theta-Y \sin \theta-x_0)^2}{a^2}+\frac{(X \sin \theta+ Y \cos \theta-y_0)^2}{b^2}=1$$
And here is the equation for your ellipse. You still need to find $x_2,y_2$ and $\cos \theta$ - I leave it to you.
See http://en.wikipedia.org/wiki/Rotation_matrix for more about rotation.
Note, that there is four possible green points you can choose, so there is four different equations you might need to check, not two.
Best Answer
From
$$ \left(\frac xa\right)^2+\left(\frac yb\right)^2-1=0 $$
changing of coordinates by rotating $-\theta$ we have
$$ \frac{(X \cos (\theta )+Y \sin (\theta ))^2}{a^2}+\frac{(Y \cos (\theta )-X \sin (\theta ))^2}{b^2}-1=0 $$
or
$$ \left(\frac{\cos ^2(\theta )}{a^2}+\frac{\sin ^2(\theta )}{b^2}\right)X^2+\left(\frac{1}{a^2}-\frac{1}{b^2}\right) \sin (2 \theta )XY+\left(\frac{\sin ^2(\theta )}{a^2}+\frac{\cos ^2(\theta )}{b^2}\right)Y^2-1=0 $$
and the normal vector $\vec n$ is given by
$$ \vec n = \left\{ \begin{array}{c} \frac{ \cos ^2(\theta ) (Y \tan (\theta )+X)}{a^2}+\frac{ \sin (\theta ) (X \sin (\theta )-Y \cos (\theta ))}{b^2} \\ \frac{ \sin (\theta ) (X \cos (\theta )+Y \sin (\theta ))}{a^2}+\frac{ \cos (\theta ) (Y \cos (\theta )-X \sin (\theta ))}{b^2} \\ \end{array} \right. $$
now changing to polar coordinates $$ E(t)=\cases{ X = \rho(t)\cos t\\ Y = \rho(t)\sin t } $$
we have
$$ \cases{ \rho ^2(t) \left(\frac{\cos ^2(t-\theta )}{a^2}+\frac{\sin ^2(t-\theta )}{b^2}\right)-1=0\\ \vec n(t) = \left\{ \begin{array}{c} \frac{\rho(t) \left(\left(b^2-a^2\right) \cos (t-2 \theta )+\left(a^2+b^2\right) \cos (t)\right)}{a^2 b^2} \\ \frac{\rho(t) \left(\left(a^2+b^2\right) \sin (t)+(a-b) (a+b) \sin (t-2 \theta )\right)}{a^2 b^2} \\ \end{array} \right. } $$
now taking
$$ \rho(t) = \frac{ab\sqrt{2}}{\sqrt{\left(b^2-a^2\right) \cos (2 (t-\theta ))+a^2+b^2}} $$
and substituting into
$$ \mathcal{E}_r(t)=(\rho(t)\cos t,\rho(t)\sin t) + r\frac{\vec n}{\|\vec n\|} $$
where $\mathcal{E}_r(t)$ is $E(t)$ expanded by $r$, we get at
$$ \mathcal{E}_r(t)=\frac{ab\sqrt{2}}{\sqrt{\left(b^2-a^2\right) \cos (2 (t-\theta ))+a^2+b^2}}\left(\cos t, \sin t\right)+r\frac{\vec n}{\|\vec n\|} $$
where
$$ \vec n = \left\{ \begin{array}{c} \frac{\sqrt{2} \left(\left(b^2-a^2\right) \cos (t-2 \theta )+\left(a^2+b^2\right) \cos (t)\right)}{a b \sqrt{\left(b^2-a^2\right) \cos (2 (t-\theta ))+a^2+b^2}} \\ \frac{\sqrt{2} \left(\left(a^2+b^2\right) \sin (t)+(a-b) (a+b) \sin (t-2 \theta )\right)}{a b \sqrt{\left(b^2-a^2\right) \cos (2 (t-\theta ))+a^2+b^2}} \\ \end{array} \right. $$
now, if the horizontal line is given by $Y=h$, the tangent circle center is located at the solutions for
$$ (0,1)\cdot \mathcal{E}_r(t) = h + r $$
where $r$ is the circle radius.
NOTE
Assuming the parametric values $a=1,b=3,h=1,r=0.5,\theta = -\frac{\pi}{6}$ we have
$$ (0,1)\cdot \mathcal{E}_r(t) - (h + r) = \frac{3 \sin (t)}{\sqrt{2 \sqrt{3} \sin (2 t)+2 \cos (2 t)+5}}+\frac{3 \sin (t)+2 \sqrt{3} \cos (t)}{3 \sqrt{2 \sqrt{3} \sin (2 t)+2 \cos (2 t)+5} \sqrt{\frac{4 \left(3 \sin (t)+2 \sqrt{3} \cos (t)\right)^2}{9 \left(2 \sqrt{3} \sin (2 t)+2 \cos (2 t)+5\right)}+\frac{4 \left(2 \sqrt{3} \sin (t)+7 \cos (t)\right)^2}{9 \left(2 \sqrt{3} \sin (2 t)+2 \cos (2 t)+5\right)}}}-\frac{3}{2}=0 $$
to solve for $t$. Follows a plot shoving in blue $E(t)$ in dashed light brown $\mathcal{E}_{\frac 12}(t)$ also in dashed red $Y=1+\frac 12$. The solution for $t$ can be obtained by binary search.
Follows a python script to find the solution angles and circle centers.
NOTE
The
binary_search
procedure needs that the function be increasing in the searching domain: so we haveresidual
andminus_residual
.