The set $G$ gives the representation of the identity and generators of the abstract group of quaternions as elements in $SL(2,\mathbb C)$ which are also in $SU(2)$. Taking the completion of this yields the representation $Q_8$ of the quaternions presented in the question.
From the description of the symmetry group as coming from here, consider the composition of two $\pi$ rotations along the $\hat x$, $\hat y$, or $\hat z$ axis. This operation is not the identity operation on spins (that requires a $4\pi$ rotation). However, all elements of $D_2$ given above are of order 2.
This indicates that the symmetry group of the system should be isomorphic to the quaternions and $Q_8$ is the appropriate representation acting on spin states. The notation arising there for $D_2$ is probably from the dicyclic group of order $4\times 2=8$ which is isomorphic to the quaternions.
It sounds like what you're asking is: how do you construct a representation of SU(2) in terms of 3x3 matrices on a real 3-dimensional vector space? (This representation is also known as the "spin-1" representation, as it's used to describe the spin of spin-1 particles.)
The H you mention, which appears to be some kind of Hamiltonian, is irrelevant to the above question. I assume it is part of a longer homework question which isn't described fully here, so I'll ignore it.
As your teacher mentions, a simple way to construct $S_x$, $S_y$, and $S_z$ is to start with the raising and lowering operators $S_+$ and $S_-$.
If you work in the $S_z$ basis, then you know what the action of $S_z$ is on each of the 3 $S_z$ eigenstates:
$S_z \begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix} = \hbar \begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix}$
$S_z \begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix} = 0$
$S_z \begin{bmatrix}0 \\ 0 \\ 1 \end{bmatrix} = -\hbar \begin{bmatrix}0 \\ 0 \\ 1\end{bmatrix}$
So the 3x3 matrix form of $S_z$ in this basis must be:
$S_z = \hbar\begin{bmatrix}1 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & -1 \end{bmatrix}$
And you also know the actions of the raising and lowering operators on these $S_z$ eigenstates (up to an undetermined constant):
$S_+$ $\begin{bmatrix}0 \\ 0 \\ 1 \end{bmatrix} = c\begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix}$
$S_+$ $\begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix} = c\begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix}$
$S_+$ $\begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix} = 0$
$S_-$ $\begin{bmatrix}0 \\ 0 \\ 1 \end{bmatrix} = 0$
$S_-$ $\begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix} = c\begin{bmatrix}0 \\ 0 \\ 1 \end{bmatrix}$
$S_-$ $\begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix} = c\begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix}$
If you take those actions and write them in matrix form, you get:
$S_+$ = c$\begin{bmatrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{bmatrix}$
$S_-$ = c$\begin{bmatrix}0 & 0 & 0 \\ 1 & 0 & 0 \\ 0 & 1 & 0\end{bmatrix}$
Then, you can write down $S_x$ and $S_y$ just by taking the right linear combinations of $S_+$ and $S_-$:
$S_x = \frac{1}{2}(S_+ + S_-)$
$S_y = \frac{1}{2i}(S_+ - S_-)$
The only final step required is to determine the constant c. This can be determined by finding the eigenvalues of the $S_x$ and $S_y$ matrices. You want them to be $-\hbar$, $0$, and $\hbar$. You can accomplish this by setting $c = \hbar\sqrt{2}$.
As for the commutation relations $[J_i,J_j] = i\hbar\epsilon_{ijk} J_k$, it just means that:
$[S_x,S_y] = i\hbar S_z$
$[S_y,S_z] = i\hbar S_x$
$[S_z,S_x] = i\hbar S_y$
You can verify these directly using matrix multiplication, for example by showing that $S_x S_y - S_y S_x = i\hbar S_z$
Best Answer
You don't really derive higher-spin matrices from lower-spin ones. Rather, they are derived from the algebra of spin operators and from how they act oon the chosen spin basis.
The basis chosen is usually the basis of eigenvectors of operator $S_z$, and are often denoted $|j,m\rangle$, $2j\in\mathbb{N}$, $m\in\{-j,-j+1\dots,j\}$. Operator $S_z$ act on them as follows:
$$ S_z|j,m\rangle = \hbar m |j,m\rangle $$ We also have operators $S_+=S_x+iS_y$ and $S_-=S_x-iS_y$. Using the algebra of spin operators it can be proven that $$ S_+|j,m\rangle = \hbar \sqrt{(j-m)(j+m+1)}|j,m+1\rangle $$ $$ S_-|j,m\rangle = \hbar \sqrt{(j+m)(j-m+1)}|j,m-1\rangle $$ (try to do it yourself, or ask if you need help). These relations are enough to write down the matrices of $S_z$, $S_+$ and $S_-$ in the basis of vectors $|j,m\rangle$, and from $S_+$ and $S_-$ you can obtain $S_x$ and $S_y$.