I do not know what the sine of the angle between two vectors is. I think it may be the vector created by connecting the tips of the two vectors but I am not sure.
How do you find the sine of the angle between two vectors?
[Math] How to find the sine of the angle between two vectors
angletrigonometryvectors
Related Solutions
They are position vectors in $\mathbb{R}^3$. Two vectors describe a plane so I presume you are being asked to find the angle between the two vectors in the plane they describe.
I've added a small illustration (done in Geogebra) to show what I mean. The two points $A$ and $B$ are described by the vectors you gave. The blue plane is described by $A, B$ and the origin and the angle between these two vectors is $153^{\circ}$ (not sure if the angle is properly visible). You can compute this angle via the dot product. If you used the dot product and got to $157.6^{\circ}$ there may be a rounding error of some description in your calculation. Either way, I believe this may be what your teacher is looking for. Does this help?
If $a - \sqrt{2} b$ is a unit vector, then the inner product \begin{equation*} \langle a - \sqrt{2}b, a - \sqrt{2}b \rangle = ||a - \sqrt{2} b||^{2} = 1. \end{equation*}
My suggestion would be to work out the inner product, first using bilinearity, and then using the relationship between the inner product of two vectors, their magnitudes, and their angles.
Best Answer
In general for $\mathbf{a},\mathbf{b}\in\mathbb{R}^3$, we have $$\|\mathbf{a}\times \mathbf{b}\|=\|\mathbf{a}\|\|\mathbf{b}\|\sin\theta,$$ where $\theta$ is the angle between $\mathbf{a}$ and $\mathbf{b}$.