Consider Euler angles $\alpha$, $\beta$, $\gamma$ with the intrinsic $Z$–$Y$–$Z$ convention. The effective rotation $S(\alpha, \beta, \gamma)$ can be written as:
\begin{equation}
S(\alpha, \beta, \gamma) = S_z(\gamma)S_y(\beta)S_z(\alpha).
\end{equation}
$S_i(\theta)$ denotes the rotation about $i$ axis by an angle $\theta$, where $i \in \{x, y, z\}$. According to my understanding,
\begin{equation}
S_y(\beta) = \pmatrix{\cos \beta & 0 & \sin \beta \\ 0 & 1 & 0 \\ -\sin \beta & 0 & \cos \beta }.
\end{equation}
However, "Mathematical Methods for Physicists" by George Arfken, Hans Weber and Harris uses
\begin{equation}
S_y(\beta) = \pmatrix{\cos \beta & 0 & -\sin \beta \\ 0 & 1 & 0 \\ \sin \beta & 0 & \cos \beta }
\end{equation}
for calculating the effective rotation $S(\alpha, \beta, \gamma)$, stating the reason as "the signs of $\sin \beta$ have to be consistent with a cyclic permutation of the axis numbering". Could someone explain this inconsistency in the signs?
Edit: The matrices used for $S_z(\alpha)$ and $S_z(\gamma)$ in the text are,
\begin{align}
S_z(\alpha) &= \pmatrix{\cos \alpha & \sin \alpha & 0 \\ -\sin \alpha & \cos \alpha & 0 \\ 0 & 0 & 1 } \\
S_z(\gamma) &= \pmatrix{\cos \gamma & \sin \gamma & 0 \\ -\sin \gamma & \cos \gamma & 0 \\ 0 & 0 & 1 }.
\end{align}
Best Answer
Your "according to my understanding" rotation around $\hat y$, (x goes towards z when you draw the diagram, as you should), is unlike the other two, so you paved your way to dyslexia purgatory.
Normally, one chooses a consistent and memorable scheme where all three axes’ rotations are on an equal footing, either all clockwise, or all anti-clockwise, $$ (x,y,z)\to (y,z,x)\to (z,x,y), ~~\leadsto \\ \hat x \times \hat y = \hat z ,\\ \hat y \times \hat z = \hat x ,\\ \hat z \times \hat x = \hat y , $$ and all three rotations are anti-clockwise around their respective axes.
In terms of your 3x3 matrices, they might as well be connected to each other by permuting the indices (1,2,3) of them cyclically, as in Arfken et al.
(This corresponds to an 120 degree rotation, anti clockwise, around the axis $\hat x + \hat y+ \hat z$.)