Given $p_1,p_2,p_3 \in S^2 = \{x \in \mathbb{R}^3 \mid \lVert x \rVert = 1\}$ all distinct, I wish to prove that
\begin{align*}
J &:=(p_1p_2 – 1)^2 + (p_1p_3 – 1)^2 + (p_2p_3 – 1)^2\\
&\qquad – (p_1p_2 – p_1p_3)^2 – (p_1p_3 – p_2p_3)^2 – (p_2p_3 – p_1p_2)^2\\
&> 0
\end{align*}
where $p_ip_j := p_i\cdot p_j$ denotes (for better readability and to save space) the regular dot product in $\mathbb{R}^3$.
$p_i \in S^2$ implies $p_ip_j \in [-1, 1)$. I can make $J \to 0^+$ by choosing $p_1 = -p_2$ and having $p_3 \to p_1$, so that $p_1p_2 = -1$, $p_1p_3 \to 1$, $p_2p_3 \to -1$. $J$ then approaches $(-2)^2+0^2+(-2)^2-(-2)^2-2^2-0^2=0$. If each $p_ip_j$ were allowed to be chosen freely in that range, $J$ can become negative, e.g. by choosing $p_1p_2=-1, p_1p_3=0, p_2p_3=0.9$. Then $J=-0.41<0$.
I believe the interrelatedness between the $p_i$’s make the inequality $J>0$ hold, but at the same time haven’t been able to show that no combination of $p_i$’s can produce e.g. those values of $p_ip_j$ that fail. (I believe $J=0$ is only possible by allowing $p_i = p_j$, but haven’t checked this.)
I tried bounding $p_1p_3$ after fixing the other two (e.g. if they were all in the $x-y$ plane, $\theta_{ij}$ are related, and $p_ip_j=\cos\theta_{ij}$) but didn’t get very far even with that simplification.
For some context in case it turns out to be useful, or just as an aside: $J$ is one of several discriminants used to classify quadratic curves. Let the plane passing through $p_1, p_2, p_3$ be $H = \{p_1 + s(p_2-p_1) + t(p_3-p_1) \mid s,t \in \mathbb{R}\}$. My $J$ arises from the intersection of $S^2$ with $H$, which is well known to be a circle in real space $\mathbb{R}^3$. Proving $J > 0$ shows that the corresponding solution curve for $(s,t)$ in the $s-t$ plane "configuration space" is an ellipse, which can also be easily seen to pass through $(s,t) = (0,0), (1,0), (0,1)$.
Best Answer
Consider the bivector
$$B=p_1\wedge p_2+p_2\wedge p_3+p_3\wedge p_1.$$
We'll use the standard inner product on the space of bivectors, defined by $\langle a\wedge b,\;c\wedge d\rangle=\langle a,c\rangle\langle b,d\rangle-\langle a,d\rangle\langle b,c\rangle$ (for arbitrary vectors $a,b,c,d$). As with any inner product, we have $\langle B,B\rangle=\lVert B\rVert^2>0$ unless $B=0$.
$$\lVert B\rVert^2=\lVert p_1\wedge p_2\rVert^2+\lVert p_2\wedge p_3\rVert^2+\lVert p_3\wedge p_1\rVert^2 \\ +2\langle p_1\wedge p_2,p_2\wedge p_3\rangle+2\langle p_2\wedge p_3,p_3\wedge p_1\rangle+2\langle p_3\wedge p_1,p_1\wedge p_2\rangle$$ $$=\Big(\lVert p_1\rVert^2\lVert p_2\rVert^2-\langle p_1,p_2\rangle^2\Big) \\ +\Big(\lVert p_2\rVert^2\lVert p_3\rVert^2-\langle p_2,p_3\rangle^2\Big) \\ +\Big(\lVert p_3\rVert^2\lVert p_1\rVert^2-\langle p_3,p_1\rangle^2\Big) \\ +2\Big(\langle p_1,p_2\rangle\langle p_2,p_3\rangle-\langle p_1,p_3\rangle\lVert p_2\rVert^2\Big) \\ +2\Big(\langle p_2,p_3\rangle\langle p_3,p_1\rangle-\langle p_2,p_1\rangle\lVert p_3\rVert^2\Big) \\ +2\Big(\langle p_3,p_1\rangle\langle p_1,p_2\rangle-\langle p_3,p_2\rangle\lVert p_1\rVert^2\Big)$$ $$=J.$$
This shows that $J\geq0$.
Now suppose $J=0$; this implies $B=0$. By the properties of the wedge product, we have
$$B=p_1\wedge p_2+p_2\wedge p_3+p_3\wedge p_1$$ $$=(p_1-p_2)\wedge(p_2-p_3).$$
Thus $B=0$ if and only if either $(p_1-p_2)=0$, or $(p_2-p_3)=0$, or $(p_1-p_2)$ and $(p_2-p_3)$ are collinear. The first two cases are ruled out since you said $p_1,p_2,p_3$ are distinct. Thus $(p_1-p_2)=c(p_2-p_3)$ for some scalar $c$, and so
$$p_1=p_2+(-c)(p_3-p_2)$$
which shows that $p_1$ is on the line connecting $p_2$ and $p_3$. This is impossible since a line intersects a sphere in at most two points (which follows easily from the quadratic formula / fundamental theorem of algebra).