I have some points that lie on the unit sphere in $\mathbb{R}^{n – 1}$, and I want to rotate them in $\mathbb{R}^n$ so that they lie on the plane defined by $\sum_{i=1}^n x_i = 0$, where $x_i$ are the coordinates of the points in $\mathbb{R}^n$.
My main question is how do I do this in $\mathbb{R}^n$, but at the moment, I can't even get it working in $\mathbb{R}^3$. Here is my $\mathbb{R}^3$ attempt:
$u_1 = (0, 0, 1)^T$ is unit normal to the $x-y$ plane.
$u_2 = (1, 1, 1)^T$ is normal to $x + y + z = 0$. Define the angle of rotation $\theta$ by $\cos(\theta) = (u_1 \cdot u_2)/|u_2|$ and the axis of rotation $a$ by $a = (u_1 \times u_2)/|u_2|$. Then apply the rotation matrix $R$ given on wikipedia. This is essentially the answer given here.
I was expecting this rotation matrix to rotate one plane to another, then if I got the planes the wrong way around I would negate the angle. But I don't get either plane rotating to the other:
$$
R\cdot(1, 1, 0)^T = (0.58, 0.58, -0.94)^T\\
R\cdot(1, -0.5, -0.5)^T = (0.56, -0.74, -0.52)^T.
$$
Any help in 3 dimensions or $n$ dimensions very much appreciated.
Best Answer
Let's go a bit more general and consider two hyperplanes $H,H'\subset \mathbb R^n$, where by hyperplane I mean a linear subspace of dimension $n-1$. Assuming that $H\ne H'$, we're going to rotate $H$ into $H'$. For that purpose, we choose unit normal vectors $\nu,\nu'\in S^{n-1}$ for $H$ and $H'$. Since rotations are orthogonal, it is enough to find one that takes $\nu$ to $\nu'$.
Since $H\ne H'$, the vectors $\nu$ and $\nu'$ are linearly independent and span a plane $E$ in $\mathbb R^n$. The vectors $f_1=\nu$ and $f_2= \frac{\nu'-\langle\nu,\nu'\rangle \nu}{|\nu'-\langle\nu,\nu'\rangle \nu|}$ are an orthonormal basis for $E$. This gives us a linear isometry $$\mathbb R^n= E\oplus E^\perp \cong \mathbb R^2\times E^\perp$$ with $\nu$ and $\nu'$ corresponding to $(1,0;0)$ and $(a,b;0)$, respectively, where $\nu'=a f_1+bf_2$. (I'll leave it to you to work out formulas for $a$ and $b$). The problem is now reduced to finding a rotation of $\mathbb R^2$ that takes $(1,0)$ to $(a,b)$. But that's easy. Since $1=|\nu'|^2=a^2+b^2$ we have $$\begin{pmatrix}a & -b\\b&a\end{pmatrix}\in SO(2)$$ Working backwards, note that $(-b,a;0)$ corresponds to $-bf_1+af_2\in\mathbb R^n$. We now obtain our desired rotation of $\mathbb R^n$ by mapping $$\nu=f_1\mapsto af_1+bf_2=\nu', \quad f_2\mapsto -bf_1+af_2$$ and leaving all vectors in $E^\perp$ unchanged.
If you insist on a matrix representation of this rotation, you'd have to extend $f_1,f_2$ to an orthonormal basis of $\mathbb R^n$ by choosing an orthonormal basis $f_3,\dots,f_n\in E^\perp$. Then the matrix $F$ with columns $f_1,\dots,f_n$ is orthogonal and the desired roation matrix is $$R=F \begin{pmatrix}\begin{smallmatrix}a & -b \\ b&a \end{smallmatrix} & 0 \\ 0 & E_{n-2} \end{pmatrix}F^t\in SO(n)$$ where $E_{n-2}$ is the unit matrix in dimension $n-2$.
In concrete cases such as $\nu = (0,\dots,0,1)$ and $\nu'=\tfrac{1}{\sqrt n}(1,1,\dots,1)$ it is easy to write down a concrete matrix $F$ as above. Again, I'll leave this to you.