For any point inside of a tetrahedron, it is possible to describe it in 3 barycentric parameters (the 4th can be deduced from the other 3). Is this reduction from 4 to 3 coordinates also possible for any other point outside the tetrahedron? If so, how do I find which 3 barycentric coordinates describe this point and how I can deduce the 4th?
Reduce 4 barycentric coordinates to 3, outside and inside of tetrahedron
barycentric-coordinates
Related Solutions
One basic approach to check if a point is inside a triangle is what's called the dot-product method (the third method described here). However this method relies somewhat on you having a right oriented coordinate system in two dimensions, but with barycentric coordinates we have three coordinates.
Now if we assume that we require the barycentric coordinates to add up to $1$ we would write the coordinates of a point $R$ as $R$ = $r_aA+r_bB+r_cC$ (avoiding having to divide by $r_a+r_b+r_c$ as required otherwise). This is awfully much like a normal coordinate system in three dimensions, and the requirement that the barycentric coordinates to add up to $1$ is the equation of a plane.
To mimic the dot-product method we need to construct a vector perpendicular to the oriented edges, but for a plane given in this form we know that we can form such a vector by forming the vector product with the normal to the plane. If $A$, $B$ and $C$ was an orthogonal base we would use the normal formula. However the fact that $R$ is inside the triangle is invariant under linear mapping - that is we can use the same formula anyway.
Now for a vector $(x,y,z)$ in that plane we form that normal by vector multiplying with a normal $(1,1,1)$ to the plane. That is
$$(x,y,z)\times(1,1,1) = (y-z , z-x , x-y)$$
So you use this to form the normals to $\overline{AB'}$, $\overline{B'C'}$ and $\overline{C'A}$ and then take the dot product of these with $\overline{AR}$, $\overline{B'R} and $\overline{C,R}$ respectively.
(note that when subtracting two points as in $\overline{AB'}$ you get a tripple, but these are not the barycentric coordinates for any point).
The barycentric coordinates $(u,v,w)$ for point $\vec{p}$ with respect to the triangle $(\vec{a},\vec{b},\vec{c})$ satisfies $$ u\vec{a}+v\vec{b}+w\vec{c} = \vec{p}, \hspace{1cm} u+v+w = 1.$$ Since $u = 1-v-w$, the first equation becomes $$ (1-v-w)\vec{a} + v\vec{b} + w\vec{c} = \vec{p}\implies v(\vec{b}-\vec{a}) + w(\vec{c}-\vec{a}) = \vec{p} - \vec{a}\implies v\vec{v_0} + w\vec{v_1} = \vec{v_2} $$ where $\vec{v_0} = \vec{b}-\vec{a}$, $\vec{v_1} = \vec{c}-\vec{a}$, and $\vec{v_2} = \vec{p}-\vec{a}$. For $0\le i,j\le 2$, define $d_{ij} = \vec{v_i}\cdot\vec{v_j}$. If we take the equation $v\vec{v_0} + w\vec{v_1} = \vec{v_2}$ and dot both sides by $\vec{v_0}$, we get $$ v\vec{v_0}\cdot\vec{v_0} + w\vec{v_0}\cdot\vec{v_1} = \vec{v_0}\cdot\vec{v_2}\implies d_{00}v+d_{01}w = d_{02}. $$ Similarly, dotting both sides by $\vec{v_1}$, we obtain $$ v\vec{v_1}\cdot\vec{v_0} + w\vec{v_1}\cdot\vec{v_1} = \vec{v_1}\cdot\vec{v_2} \implies d_{01}v+d_{11}w = d_{12}. $$ Thus we have a system of equations \begin{align} d_{00}v + d_{01}w &= d_{02} \\ d_{01}v + d_{11}w &= d_{12} \end{align} whose solution is $$ v = \frac{d_{11}d_{02}-d_{01}d_{12}}{d_{00}d_{11}-d_{01}^2} \hspace{1cm} w = \frac{d_{00}d_{12}-d_{01}d_{02}}{d_{00}d_{11}-d_{01}^2}. $$ This is the formula given in the code.
As a side note, the denominator $d_{00}d_{11}-d_{01}^2$ is always nonnegative by Cauchy-Schwarz, and it is always positive as long as the triangle is nondegenerate.
Best Answer
The three-dimensional barycentric coordinates of a point $P$, with respect of tetrahedron $ABCD$, are either:
We can generalize either of these definitions to points outside the tetrahedron if we use a signed volume. There are many equivalent definitions. For example, assuming that the signed volume of $ABCD$ is positive, we can define the signed volume of $ABCP$ to be positive if $P$ is on the same side of plane $ABC$ as $D$, and negative if it is on the other side. Do the same for the other volumes.
Then, the un-normalized signed volumes will still sum to $[ABCD]$, and the normalized signed volumes will still sum to $1$, and so any three of the coordinates tell you the fourth under either convention.
More algebraically, the volume of tetrahedron $ABCD$ is given by the formula $$ \frac16 \det\begin{bmatrix}1 & x_a & y_a & z_a \\ 1 & x_b & y_b & z_b \\ 1 & x_c & y_c & z_c \\ 1 & x_d & y_d & z_d\end{bmatrix}. $$ Usually, we take the absolute value, but if we don't, then we get one notion of signed volume. Then the coordinates for point $P$ are determined by the four volumes given by the four determinants
$$ \scriptsize \left(\frac16 \det\begin{bmatrix}1 & x_a & y_a & z_a \\ 1 & x_b & y_b & z_b \\ 1 & x_c & y_c & z_c \\ 1 & x_p & y_p & z_p\end{bmatrix}, \frac16 \det\begin{bmatrix}1 & x_a & y_a & z_a \\ 1 & x_b & y_b & z_b \\ 1 & x_p & y_p & z_p \\ 1 & x_d & y_d & z_d\end{bmatrix}, \frac16 \det\begin{bmatrix}1 & x_a & y_a & z_a \\ 1 & x_p & y_p & z_p \\ 1 & x_c & y_c & z_c \\ 1 & x_d & y_d & z_d\end{bmatrix}, \frac16 \det\begin{bmatrix}1 & x_p & y_p & z_p \\ 1 & x_b & y_b & z_b \\ 1 & x_c & y_c & z_c \\ 1 & x_d & y_d & z_d\end{bmatrix}\right). $$ We know that the four volumes add up to the volume of $ABCD$ inside the tetrahedron, because the four tetrahedra $ABCP, ABPD, APCD, PBCD$ partition $ABCD$. But the sum of these four determinants is a polynomial in $x_p, y_p, z_p$ (taking the points $A,B,C,D$ as constant). If this polynomial has a constant value for all $(x_p, y_p, z_p)$ inside the tetrahedron, then it must be constant on all of $\mathbb R^3$, which is why the signed volumes continue to behave outside the tetrahedron.