If I have an ellipsoid in three dimensions (actually an oblate spheroid, the Earth).
$$\frac{x^2+y^2}{a^2} + \frac{z^2}{b^2} = 1$$
And a ray represented by the point $P = (x_p, y_p, z_p)$ with direction vector $D = (x_d, y_d, z_d)$
I can form an ellipse by intersecting the plane formed by the ray and the origin with the ellipsoid. How can I find the points on that ellipse (there should be two I believe), where the normal vector is perpendicular to the ray?
I'm effectively looking for the furthest points on the ellipse that are "visible" to the origin of the ray, $O$. Since there's two solutions, I'd like the one closer to the ray.
Here's a picture that I hope clarifies, after forming an ellipse by intersecting a plane with the ellipsoid, I want to find point A where the normal vector is perpendicular to the line containing the point and the ray origin.
I believe this is equivalent to finding the tangent line that contains the point P.
Best Answer
The plane through the ray and the origin intersects your ellipsoid to form a planar ellipse. To find this planar ellipse one must establish a basis vector on the plane such that any point on the plane can be represented by two parameters ($\alpha$ and $\beta$). $$ \vec{r} = \alpha \vec{p} + \beta \vec{d} $$
The ellipse on the plane is now given by the equation
$$ \alpha^2 C_{11} + 2 \alpha\beta C_{12} + \beta^2 C_{22} = 1 $$
$$ \pmatrix{\alpha \\ \beta \\ 1}^\top \pmatrix{C_{11} & C_{12} & \\ C_{12} & C_{22} & \\ & & -1} \pmatrix{\alpha \\ \beta \\ 1} =0 $$
where $$\begin{align} C_{11} &= \frac{x_p^2}{a^2} + \frac{y_p^2}{a^2} + \frac{z_p^2}{b^2} \\ C_{22} &= \frac{x_d^2}{a^2} + \frac{y_d^2}{a^2} + \frac{z_d^2}{b^2} \\ C_{12} &= \frac{x_p x_d}{a^2} + \frac{y_p y_d}{a^2} + \frac{z_p z_d}{b^2} \end{align} $$
Now for the fun part. To be perpendicular to the ray the point $\vec{r}$ must obey the projection $$ \vec{d}^\top (\vec{r}-\vec{p}) =0 $$
From the expansion of $\vec{r}$ in terms of the planar coordinates $\alpha$ and $\beta$ this projection is used to solve for
$$ \alpha = 1- \frac{\vec{d}^\top \vec{d}}{\vec{d}^\top \vec{p}} \beta = 1-\frac{x_d^2+y_d^2+z_d^2}{x_d x_p + y_d y_p + z_d z_p} \beta = 1- \lambda \, \beta $$
So given any distance along the ray $\beta$ the 3D position of the point is $\vec{r} = (1-\lambda \,\beta) \vec{p} + \beta \vec{d} $ where $\lambda$ is the fixed ratio $\lambda = \frac{x_d^2+y_d^2+z_d^2}{x_d x_p + y_d y_p + z_d z_p}$.
The points on the ellipse that are perpendicular to the ray solve the equation
$$ (1-\lambda\,\beta)^2 C_{11} + 2 (1-\lambda\,\beta)\beta C_{12} + \beta^2 C_{22} = 1 \\ (C_{11} \lambda^2 -2 C_{12} \lambda + C_{22}) \beta^2 + 2 (C_{12}-C_{11} \lambda) \beta + (C_{11-1}) = 0 $$
This has two solutions (as expected)
$$ \beta = \frac{C_{11} \lambda-C_{12} \pm \sqrt{ C_{11} (\lambda^2-C_{22}) + C_{12}^2 -2 C_{12} \lambda + C_{22}}}{C_11 \lambda^2 -2 C_{12} \lambda + C_{22}} $$
And the back substitution
$$ \alpha = 1 - \lambda\, \beta \\ \vec{r} = \alpha \vec{p} + \beta \vec{d} $$
Edit 1 To find the lines tangent to the planar ellipse, and passing through P you do the following:
The polar line intersects the ellipse at the tangent points $Q_1$ and $Q_2$. To find these points form the equation of the line $C_{11} \alpha + C_{12} \beta - 1 =0$ and plug it into the ellipse equation to find $$Q_1 = \pmatrix{ 1 + \tfrac{C_{12} \sqrt{C_{11}-1}}{K}, & - \tfrac{C_{11} \sqrt{ C_{11}-1 }}{K}, & C_{11} }$$ $$Q_2 = \pmatrix{ 1 - \tfrac{C_{12} \sqrt{C_{11}-1}}{K}, & \tfrac{C_{11} \sqrt{ C_{11}-1 }}{K}, & C_{11} }$$ where $K=\sqrt{C_{11} C_{22}-C_{12}^2}$.
The tangent line coordinates are $T_1 = C \,Q_1$ and $T_2 = C \, Q_2$ $$ T_1 = \left[ \matrix{ C_{11}, & C_{12} -K \sqrt{C_{11}-1} , & -C_{11} } \right] $$ $$ T_2 = \left[ \matrix{ C_{11}, & C_{12} +K \sqrt{C_{11}-1} , & -C_{11} } \right] $$
In equation form the tangent lines are $$ (C_{11}) \alpha + (C_{12} - K \sqrt{C_{11}-1}) \beta - C_{11} = 0 $$ $$ (C_{11}) \alpha + (C_{12} + K \sqrt{C_{11}-1}) \beta - C_{11} = 0 $$