The $8$ matrices in $K$ with determinant $1$ represent rotations. $-I\in K$ represents an inversion. The other $7$ elements represent reflections ($5$) and improper rotations ($2$), depending on if $tr(A)=1$ (reflection) or $-1$ (improper rotation).
By $4$-prism I think they mean a square cuboid. They define it as "a box with square ends that is not a cube". So we can label the vertices of the opposite squares as $1,2,3,4$ and $5,6,7,8$. We can define a counter-clockwise rotation $f:\{1234\}\to \{4123\}$ and another rotation $g:\{1234\}\to \{5876\}$. We can see that $|f|=4$ and $|g|=2$, so the rotations can be represented as $\langle f,g \rangle$. Is this correct? I'm really not sure about this question or how to represent the reflections/improper rotations in terms of $D_4$. And where does $\mathbb{Z}_2$ enter the equation?
Best Answer
Consider the rotational symmetries of the square cuboid. If we define the longitudinal axis to be the y-axis, then there is a rotation about the y-axis of order $4$, that is the rotation of $90$ degrees, this has the following elements: $$R_0, R_{90}, R_{180}, R_{270}$$ Their matrices are represented by the unitary orthogonal $det = 1$ matrices with the eigenvector $(0, 1, 0)$ corresponding to the eigenvalue $1$.
Then there is a rotation going through the x and z axis, each of order $2$ and 180 degrees in both directions, this yields 4 rotations of order $2$. Again, this corresponds to the orthogonal matrices with eigenvalue $1$ and corresponding eigenvector $(1, 0, 0)$ and $(0, 0, 1)$. We have just counted $4+4$ rotations, and four of them are in the form of $R_0, R_{90}, R_{180}, R_{270}$ and the other four are of order $2$... sound familiar?
Perhaps not very coincidentally, this subgroup of rotations is actually a copy of $D_4$ (check by the presentation definition of a dihedral group, that is it can be generated by two involutions), such that: $$D_4 = <a, b \ | \ a^n = b^2 = I, (ab)^2 = I>$$
$D_4$ is normal in the symmetries of the square cuboid (since it has index $2$, used by the assumption given in the question before, that is has $12$ isometries). Noting that the entire symmetry group can be composed by all the rotations and a single reflection, consider the point of inversion $-I$. It is trivial to prove that $\{I, -I\}$ is normal as well. (noting that $-I^{-1} = -I$).
What have just showed? We have showed that the symmetries can be composed via two normal subgroups with only the trivial intersection, the rotations being isomorphic to $D_4$ and the set $\{I, -I\}$ being isomorphic to $\mathbb{Z}_2$ (you can check this manually be reviewing the subgroup of matrices).
Indeed the symmetries of the 4-prism is isomorphic to $D_4 \oplus \mathbb{Z}_2$
Additionally, more simply is that once you have composed the entire rotation subgroup in the square cuboid, you know that it is symmetric about the origin (the tetrahedron defies this). Since it is symmetric about the origin, that means you can take the direct product with $\mathbb{Z}_2$. This immediately yields the intended isomorphism.