OK. Let's say that $(v_1, v_2)$ is the first vector-pair (your "oriented vector"). I'm going to change that to $(v_1, v_2, v_1 \times v_2)$, which is a triple of unit-length, orthogonal unit vectors, and in fact, the basis $(v_1, v_2, v_1 \times v_2)$ is positively oriented (i.e., can be rotated to align with the standard unit vectors $e_1, e_2,$ and $e_3$, in the $x$-, $y$-, and $z$-directions, respectively, in that order). Let $V$ be the matrix whose columns are $v_1$, $v_2$, and $v_1 \times v_2$. Then
$$
T_V : \mathbb R^3 \to \mathbb R^3 : v \mapsto Vv
$$
is a linear transformation taking $e_1$ to $v_1$, $e_2$ to $v_2$, and $e_3$ oto $v_1 \times v_2$. Clear so far?
Build a similar matrix $W$ for your other set of vectors (extneded by the cross-product).
Now consider the transformation
$$
S : : \mathbb R^3 \to \mathbb R^3 :v \mapsto W V^t v=W V^{-1} v .
$$
It takes the vector $v_1$ to $e_1$ (when you multiply by $V^{-1}$) and then to $w_1$ (when you multiply by $W$); similarly, it takes $v_2$ to $w_2$. And it also happens to take $v_1 \times v_2$ to $w_1 \times w_2$.
And it's easy to build, because rather than computing $V^{-1}$, you can compute $V^t$, because the unit-ness and orthogonality of the $v$-vectors means that $V^t = V^{-1}$.
Thanks to David K and Rollen comments I was able to find a solution:
I only had $v_{Ax}$ and $v_{Bx}$ but I could calculate $v_{Ay}$ and $v_{By}$ using other information about the rigid body (i.e. the positions of two aligned points of the rigid body) I constructed the rotation matrices for the two frames using these two direction vectors and the cross product between them.
$R_A=\begin{bmatrix}v_{Ax}&v_{Ay}&v_{Ax}\times v_{Ay}\end{bmatrix}$;
$R_B=\begin{bmatrix}v_{Bx}&v_{By}&v_{Bx}\times v_{By}\end{bmatrix}$
and then the rotation matrix from A to B is given by:
$R = R_B \cdot R_A^T$
Thank you for your help.
Best Answer
We are looking for a vector $\mathbf{v}$ such that the angle between $\mathbf{v}$ and $\mathbf{x}_2 - \mathbf{x}_1$ before rotation of $\mathbf{x}_1$ and $\mathbf{x}_2$, which is $$ \arccos\left(\frac{\mathbf{v} \cdot (\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{x}_2 - \mathbf{x}_1||}\right), $$ is equal to the angle between $\mathbf{v}$ and $\mathbf{x}_2 - \mathbf{x}_1$ after rotation of $\mathbf{x}_1$ and $\mathbf{x}_2$, which is $$ \arccos\left(\frac{\mathbf{v} \cdot (\mathbf{R}\mathbf{x}_2 - \mathbf{R}\mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{R}\mathbf{x}_2 - \mathbf{R}\mathbf{x}_1||}\right), $$ where $\mathbf{R}$ is a rotation matrix. Equating these two expressions yields \begin{align} \arccos\left(\frac{\mathbf{v} \cdot (\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{x}_2 - \mathbf{x}_1||}\right) &= \arccos\left(\frac{\mathbf{v} \cdot (\mathbf{R}\mathbf{x}_2 - \mathbf{R}\mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{R}\mathbf{x}_2 - \mathbf{R}\mathbf{x}_1||}\right) \\ \\ \frac{\mathbf{v} \cdot (\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{x}_2 - \mathbf{x}_1||} &= \frac{\mathbf{v} \cdot (\mathbf{R}\mathbf{x}_2 - \mathbf{R}\mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{R}\mathbf{x}_2 - \mathbf{R}\mathbf{x}_1||} \\ \\ \frac{\mathbf{v} \cdot (\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{x}_2 - \mathbf{x}_1||} &= \frac{\mathbf{v} \cdot \mathbf{R}(\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{R}(\mathbf{x}_2 - \mathbf{x}_1)||} \end{align} Since $||\mathbf{x}|| = \sqrt{\mathbf{x}\cdot\mathbf{x}}$, and since the dot product is invariant to rotation, then \begin{align} \frac{\mathbf{v} \cdot (\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{x}_2 - \mathbf{x}_1||} &= \frac{\mathbf{v} \cdot \mathbf{R}(\mathbf{x}_2 - \mathbf{x}_1)}{||\mathbf{v}|| \times ||\mathbf{x}_2 - \mathbf{x}_1||} \\ \\ \mathbf{v} \cdot (\mathbf{x}_2 - \mathbf{x}_1) &= \mathbf{v} \cdot \mathbf{R}(\mathbf{x}_2 - \mathbf{x}_1) \end{align} Again, since the dot product is invariant to rotation, then this equation is true when:
In other words, the angle between $\mathbf{x}_1$ or $\mathbf{x}_2$ and $\mathbf{x}_2 - \mathbf{x}_1$ is invariant to rotation.