If $\mathbf{a}$ and $\mathbf{b}$ are two vectors, we get the magnitude of the rotation or moment as $|\mathbf{a}||\mathbf{b}|\sin(\angle ab)$.
Now, we are multiplying that with the unit vector orthogonal to the $\mathbf{ab}$-plane. That's when we get $\mathbf{a}\times \mathbf{b}$ vector.
But my question is, why normal to the $\mathbf{ab}$-plane? Why not in any other direction? Why 90 degree and nothing other than that?
Best Answer
There are a few different ways to see this. If you want to define the cross product in $\mathbb R^3$ in a coordinate-free, geometric manner you define $v \times w$ to be equal to the vector with magnitude $\| v \| \cdot \| w\| \sin(\theta)$ (with $\theta$ the angle between $v$ and $w$) pointing in the direction determined by the right-hand rule. The usual coordinate description is then a consequence of this.
Orthogonality can be deduced from the coordinate description of the cross product via the triple scalar product. First recall that two non-zero vectors are orthogonal if and only if their inner product is zero. For $v, w \in \mathbb R^3$, also recall that $v \cdot (v \times w) = \det (v, v, w)$ and $w \cdot (v \times w) = \det (w, v, w)$. Here, $(v, v, w)$ and $(w, v, w)$ are taken to be $3 \times 3$-matrices whose rows are given by the indicated vectors. Both determinants are $0$ since the matrices have two identical rows and are hence singular. This shows that both $v$ and $w$ are orthogonal to $v \times w$, so that $v \times w$ is parallel to the normal vector determined by the span of $v$ and $w$.