Imagine that we line up the students at random, and assign the first $4$ to one group, the next $4$ to another, and the last $4$ to another.
We can analyze as follows. We have $3$ copies of the letter $H$, and $9$ copies of the letter $M$. We make a $12$-letter word. What is the probability there will be an $H$ among the first $4$ letters, and an $H$ among the next $4$, and an $H$ among the last $4$?
There are $\binom{12}{3}$ equally likely ways to place the $3$ $H$'s.
The number of choices with one $H$ in the first $4$, another $H$ in the next $4$, another in the next $4$ is $4^3$.
Our required probability is therefore $\dfrac{4^3}{\binom{12}{3}}$.
Remark: Your number $\binom{12}{8}\binom{8}{4}\binom{4}{4}$ counts the number of ways to divide our $12$ people into three uniformed teams, the Blues, the Whites, and the Reds. That's fine.
Now we count the number of ways to divide into uniformed teams with an Honours student on each team. The Honours student of the Blues can be chosen in $\binom{3}{1}$ ways. The rest of the team can be chosen in $\binom{9}{3}$ ways. For every such choice, the Honours student in the Whites can be chosen in $\binom{2}{1}$ ways, and the rest of her team in $\binom{6}{3}$ ways. Now we are finished, although we can tack on a decorative $\binom{1}{1}\binom{3}{3}$ for a total of $\binom{3}{1}\binom{9}{3}\binom{2}{1}\binom{6}{3}\binom{1}{1}\binom{3}{3}$.
Now for the probability, divide.
$12\choose 6$ double counts. For instance you could choose A, B, C, D, E, F for one group, leaving G, H, I, J, K, L for the other; or you could choose G, H, I, J, K, L for one group, leaving A, B, C, D, E. F. But this actually gives you the same two groupings.
Best Answer
You are very close. The difference between (i) and (ii) is that the order of the four groups no longer matters (since they are all studying the same topic), so you need to divide by another 4!. Hence the answer to (ii) is $\dfrac{\binom{16}{4}\binom{12}{4}\binom{8}{4}}{4!}$.