Calculators either use the Taylor Series for $\sin / \cos$ or the CORDIC algorithm. A lot of information is available on Taylor Series, so I'll explain CORDIC instead.
The input required is a number in radians $\theta$, which is between $-\pi / 2$ and $\pi / 2$ (from this, we can get all of the other angles).
First, we must create a table of $\arctan 2^{-k}$ for $k=0,1,2,\ldots, N-1$. This is usually precomputed using the Taylor Series and then included with the calculator. Let $t_i = \arctan 2^{-i}$.
Consider the point in the plane $(1, 0)$. Draw the unit circle. Now if we can somehow get the point to make an angle $\theta$ with the $x$-axis, then the $x$ coordinate is the $\cos \theta$ and the $y$-coordinate is the $\sin \theta$.
Now we need to somehow get the point to have angle $\theta$. Let's do that now.
Consider three sequences $\{ x_i, y_i, z_i \}$. $z_i$ will tell us which way to rotate the point (counter-clockwise or clockwise). $x_i$ and $y_i$ are the coordinates of the point after the $i$th rotation.
Let $z_0 = \theta$, $x_0 = 1/A_{40} \approx 0.607252935008881 $, $y_0 = 0$. $A_{40}$ is a constant, and we use $40$ because we have $40$ iterations, which will give us $10$ decimal digits of accuracy. This constant is also precomputed1.
Now let:
$$ z_{i+1} = z_i - d_i t_i $$
$$ x_{i+1} = x_i - y_i d_i 2^{-i} $$
$$ y_i = y_i + x_i d_i 2^{-i} $$
$$ d_i = \text{1 if } z_i \ge 0 \text{ and -1 otherwise}$$
From this, it can be shown that $x_N$ and $y_N$ eventually become $\cos \theta$ and $\sin \theta$, respectively.
1: $A_N = \displaystyle\prod_{i=0}^{N-1} \sqrt{1+2^{-2i}}$
When $\sin$ is defined geometrically, it is typically as the vertical coordinate on the unit circle. Certainly this is the only sensible extension from $[0^\circ, 90^\circ]$ if you want to retain any of the nice properties like analyticity, angle addition formulas, Euler's formula, etc.
Best Answer
When trigonometric functions like sine and cosine are applied to situations where we're dealing with angles that are greater than or equal to 90 degrees, the logic based purely on the right triangle definition of trigonometric functions as we know it breaks because in elementary trigonometry the sum of the angles in a right triangle (or any other triangle, for that matter) can't be greater than $180^\circ$. That's obvious.
Let's say you have a 135-degree angle ($90^\circ+45^\circ=135^\circ$), as in your picture. Basically, you still, kind of, use the right triangle idea, but you now must take into account the fact that the legs of the triangle can have negative values because the whole idea that the angle can be greater than or equal to 90 degrees comes from the unit circle which we work with in the Cartesian coordinate system. Therefore, when $x$ is negative, the length of the corresponding leg has a negative value. When $y$ is negative, the length of the corresponding leg is going to be negative as well. Does this make sense to you?
If you find my explanation confusing, here's a more-to-the-point version: forget about the right triangle definition for angles greater than or equal to 90 degrees. It doesn't really work. Now you work in the Cartesian coordinate system where the hypotenuse equals $1$, $\cos\theta$ is the x-coordinate and $\sin\theta$ is the y-coordinate.