3dellipsoidsgeometryprojective-geometryquadrics
There is an ellipsoid of unknown dimensions and unknown orientation hanging somewhere (also unknown) in $3$-dimensional space. You are given three perspective images of it from three distinct cameras whose positions and orientations are known.
How can the ellipsoid be determined based on this information?
This problem is inspired by
After translation, so that the center of the ellipse is the origin, and rotation, such that the axes of the ellipse align with the coordinate axes (these two transformations preserve the distance you are looking for ) the equation of an ellipse becomes $x'Ax=c$ where $A$ is a diagonal matrix with non-negative diagonal elements and $c$ is any constant. Then given any orientation, that is you are given the unit vector along that direction, you can decompose the unit vector according to your coordinate system, suppose it becomes $l_1e_1+l_2e_2+l_3e_3$, where $l_1,l_2,l_3$ are scalars and $e_1,e_2,e_3$ are unit vectors along the chosen coordinate axes, then the point on the surface of the ellipse that you seek, will be some scalar multiple of this vector, $d(l_1e_1+l_2e_2+l_3e_3)$ that satisfies the ellipse equation. Put this in your equation and solve for $d$. Once you get $d$ you can calulate the distance as $(d^2l_1^2+d^2l_2^2+d^2l_3^2)^{1/2}$.
As noted in a comment, the ellipse lies on the intersection of the two cones having the centers of the cameras as vertices, and the respective images as guiding curves. The intersection of two quadrics is, in general, a quartic curve. As this curve contains an ellipse, then it must be made of two conics, one of them being the ellipse we want to determine. That means there are in general two possible solutions.
To find the intersection, one could find the equations of both cones. This can be done, for instance, by taking nine points on each cone and inserting their coordinates into the general equation of a quadric. But solving the resulting system of equations can be quite tedious.
I'll suggest then a different approach. Take five points $ABCDE$ on the first image and consider the five lines $abcde$ passing through each point and the center $O_1$ of the first camera. The images of these lines, relative to the second camera, can then be constructed: they are five lines lying in the plane of the second image, each of them intersecting it at two points: $(A_1,A_2)$, $(B_1,B_2)$ and so on. If $O_2$ is the center of the second camera, we can then construct the intersection point $P_1$ between line $A_1O_2$ and line $a$, and the intersection point $P_2$ between line $A_2O_2$ and line $a$. Points $P_1$ and $P_2$ lie then on the intersection of the cones, each on a different conic.
We can similarly find the other four couples of points on the intersection: $(Q_1,Q_2)$, $(R_1,R_2)$ and so on. Five of these ten points lie on the first conic, and the others lie on the second conic, but we don't know which lies on which. To find out, we can consider all $32$ possible cases $(P_i,Q_j,R_k,\dots)$ and check if all five points lie on a the same plane: this should be true only in two cases, which are then five points on each of the two possible conics. These can then be constructed.
See here for a geometrical construction of center and axes of an ellipse, given five points.
EDIT.
You can see below an example of the construction, made with GeoGebra. Note that $O_1$ and $O_2$ are on the same side of the plane of the first solution (ellipse $P_2Q_2R_1S_1T_1$) while they are on opposite sides of the plane of the second solution (ellipse $P_1Q_1R_2S_2T_2$). If this kind of information is given one can then isolate the right solution.
Best Answer
Most of the following should apply to quadrics in general, but I'll refer to here to proper ellipsoids and not worry about the general case and edge conditions.
Before sketching out two reconstruction methods here are some preliminaries.
An ellipsoid is defined by a quadratic equation
$$ Ax^2+By^2+Cz^2+Dxy+Eyz+Fzx+Gx+Hy+Iz+J=0. $$
This can be rewritten in the form $$\mathbf{X}^T Q \mathbf{X}=0$$ where $\mathbf{X}^T=[x,y,z,1]$ and $Q=Q^T$ is a $4\times4$ symmetric matrix. If $Q=[q_{ij}]$ then $q_{11}=A, q_{12}=q_{21}=D/2$, etc.
This form is handy for various operations. For example, $\mathbf{X}^T Q^{-1} \mathbf{X}=0$ defines the quadric that is the dual of the ellipsoid defined by $\mathbf{X}^T Q \mathbf{X}=0$. (for more on this see section 2.2 of this paper.)
$5$ points in general define a conic. The well known method of finding the equation of a conic, given $5$ points is given in this answer.
$9$ points in general define an ellipsoid. To find the equation of the ellipsoid, generalize the method used for conics.
There is a caveat, in that the points have to be in some sense independent. Otherwise the equation will be degenerate.
For the case of conics, the points have to be not only distinct, but no more than three points can be collinear.
For the case of quadrics/ellipsoids, no more than $5$ of the $9$ points can lie on the same conic. Given two conics lying on the ellipsoid, no more than $8$ points can lie on these two conics (see Figure 2 in this paper).
With these preliminaries, here are some suggestions for solving the $3$-camera reconstruction problem.
Find nine points on the ellipsoid. An answer to a related question gives a method for computing points given two cameras. This method identifies points on two ellipses lying on the ellipsoid, but as noted in the preliminaries a maximum of $8$ points can be used from these ellipses. So a ninth point can be found from a different pair of cameras using the third camera.
Find nine tangent planes. To find a tangent plane, take a point $T$ on one of the image plane ellipses, and find its tangent $t$ to the ellipse. If the point $X$ is the corresponding camera position the plane containing $t$ and $TX$ is a tangent plane of the ellipsoid. Find three such tangent planes for each image ellipse, for a total of nine tangent planes. Using the nine points that are the duals of the tangent planes, build a quadric on which those points lie. Then the dual of that quadric (using the trick in the preliminaries) is the desired ellipsoid.
Use epipolar methods. This paper uses concepts from epipolar geometry (important in computer vision) to solve the problem. Frankly, I haven't had the time to read and understand it, but maybe it is helpful. I don't doubt that the solution they give is smarter and more robust than the two suggestions I've given above.