One defines the circular trigonometric functions in three stages.
Stage 1: This uses actual triangles. Then all the lengths are positive, and the ratios are of positive numbers, but angles only make sense in the interval $[0,\pi/2]$ radians. In this setting, we can place a right triangle so that one vertex is at the origin of our coordinate system, the leg it is on runs along the $x$-axis to the right angle, then the other leg runs parallel to the $y$-axis to the other vertex, and the hypotenuse is a line segment from the origin to this last vertex, lying entirely in the first quadrant. (Or, in the degenerate case where the second leg is zero, on the $x$-axis, which is not part of the first quadrant.) In this stage, we say things like "the sine of an angle is the ratio of the opposite to the hypotenuse".
Stage 2: We allow the hypotenuse to lie in any quadrant. Then we still construct right triangles ("reference triangles") in "standard position" as: put one of the vertices not at the right angle at the origin, put one leg along the $x$-axis, turn at right angles from the $x$-axis and run parallel to the $y$-axis for the other leg. This allows positive and negative values for the $x$- and $y$-coordinates of the hypotenuse and hence for the signed lengths of the first and second legs. Then we say things like "the sine of the angle is the ratio of the $y$-coordinate to the hypotenuse". (The length of the hypotenuse is always positive because it is always a length, not a coordinate.)
Stage 3: We generalize to angles less than $0$ and greater than $2\pi$ radians. This is where we get "coterminal angles". Everything still happens with right triangles, signed leg lengths, and on one copy of the coordinate plane, but we imagine that we have spun our angle around the origin many, many times in either direction. (This more accurately models real rotating systems -- a wheel doesn't rotate exactly once and then stop...)
Having made these generalizations, we are then ready to go back and apply what we have learned to non-right (degenerate, acute, and obtuse) triangles. Your language suggests you have a gap between Stage 1 and this generalization.
You are right that, given the definition of the doc product, $\vec{a} \cdot \vec{b}=ab\cos\theta$, its direct application is to find the angle between the two vectors. However, such definition has many useful properties, such as the distributive one below,
$$\vec{x}\cdot (\vec{a} - \vec{b})=\vec{x}\cdot\vec{a} -\vec{x}\cdot\vec{b} $$
which in turn allows for a lot of other applications. For example, for a triangle with vertexes $\vec{a}$, $\vec{b}$ and $\vec{c}$, we have
$$\vec{c} = \vec{a} - \vec{b}$$
One could use the dot product and its distributive property to derive the cosine rule fairly effortlessly,
$$\vec{c}\cdot\vec{c} = (\vec{a}-\vec{b})\cdot (\vec{a}-\vec{b})$$
$$c^2= a^2 -2\vec{a}\cdot \vec{b} + b^2$$
$$c^2= a^2 + b^2 -2ab\cos\theta$$
Note that the proof of the Pythagorean formula is just a special case.
The derivation of the cosine rule is a convincing example for illustrating the usefulness of the dot product. There are many other useful and important applications as well in the vector space and with Cartesian coordinates.
Best Answer
Note that $\cos$ is defined by $$\cos \theta = \frac{\text{adjacent}}{\text{hypotenuse}}$$
So we have $$\cos 15^\circ=\frac{6}{\text{hypotenuse}}\implies\text{hypotenuse}=\frac{6}{\cos 15^\circ} $$
$$\implies \text{hypotenuse} = \frac{6}{\frac{\sqrt{2}+\sqrt{6}}{4}}=6(\sqrt{6}-\sqrt{2})$$