I need a mathematical algorithm for finding the angle, formed by three points, which is open toward a fourth point.
For example, in Fig1 below I desire angle $\theta$ because it is "facing" point $P$. However, the formula for the angle between two vectors, $$\theta=\cos^{-1}\Big(\frac{\vec{u}\cdot\vec{v}}{uv}\Big)$$ gives angle $\alpha$ because the above relation always returns the angle which is less than 180 degrees.
(Fig1)
In contrast, in Fig2 I desire angle $\alpha$ because this time it is "facing" point $P$.
(Fig2)
As a final example, in Fig3 below I need angle $\theta$, because it is still technically "facing" point $P$.
(Fig3)
This problem can occur in any orientation, making it difficult to say, for example, "If point $P$ is to the left of points $A, B, C$, use the leftmost angle, otherwise use the right" or something like that.
Any help would be appreciated!
My Attempted Solution
If anyone's interested, my current solution is to split the angle facing the point $P$ in half and add those two together.
In Fig4 below, I use the grey line to split the angle in half, find $s$ and $t$ using dot product, then add $s$ and $t$ to get the angle facing $P$.
(Fig4)
This works until I reach a situation like that in Fig5. What I need is $s+t$, but what I get is $q+t$ because the dot product gives $q$ (since $s$ is greater than $180$ degrees).
(Fig5)
It's a conundrum.
Best Answer
Unless $A-B$ and $C-B$ are linearly dependent, you can uniquely express $P-B$ in terms of them: $$ P-B=r\cdot(A-B)+s\cdot(C-B).$$ Now $P$ is in the smaller of the two possible angles if and only if $r$ and $s$ are both positive. It is on the ray $BA$ if and only if $s=0$ and $r\ge0$, and on the ray $BC$ if and only if $r=0$ and $s\ge 0$. In all other cases, it is in the larger angle.