I have a feeling your teacher/professor intended for you to learn the following (we first introduce some notation):
For two sets $A, B$ in a common universe $U$, define their union as
$$
A \cup B = \{x \in U : x \in A \text { or } x \in B\}.
$$
Define their intersection as:
$$
A \cap B = \{x \in U : x \in A \text{ and } x \in B\}.
$$
It is important not to get too wrapped in the English here. Being in $A \cup B$ simply means being in one of the two sets (or possibly both). Being in $A \cap B$ simply means being in set $A$ and being in set $B$ at the same time.
Finally, a finite set $A$ with $k$ elements ($k$ things in the set) has cardinality $k$, and this is written as $|A| = k$, or $\#A = k$ or even sometimes, $n(A) = k$.
Therefore:
$$
\{1,3,5\} \cup \{1,2,3\} = \{1,2,3,5\},
$$
while
$$
\{1 ,3 , 5 \} \cap \{ 1 , 2, 3 \} = \{ 1, 3\}.
$$
Also,
$$
| \{ 1,3,5\}| = 3,
$$
while
$$
| \{ 1, 3\} | = 2.
$$
Now, what your teacher probably wanted you to learn was the following "rule":
$$
|A \cup B| = |A| + |B| - |A \cap B|.
$$
This is easy to see it is true, since to count the number of elements that are in either $A$ or $B$ (or possibly both $A$ and $B$), you count the number of elements in $A$, add to it the number of elements in $B$, and then subtract the stuff you double counted, which is precisely the elements in $A \cap B$.
Therefore, if you want to find the number of sides of a die that are (say) even or prime, you count the number of sides which are even (there are $3$ such sides - namely $2$ , $4$, $6$) add to it the number of sides which are prime (again, there are $3$ such sides - namely $2$, $3$, $5$), and then subtract the sides which we double counted (we counted the side with the number $2$ twice).
Therefore, there are $3 + 3 - 1 = 5$ sides of a die which are even or prime.
Now, you can take this strategy and count the number of cards in a deck which are either non face cards or clubs.
The "exclusion-based" formulation you're trying to use for exactly two suits works fine; you just have a minor error. You can choose the two suits in $4\choose 2$ ways, and then choose a five-card hand from the $26$ cards in those two suits in $26\choose 5$ ways. This counts a number of single-suit hands as well, which you want to exclude. In fact, each of the $4\times{13\choose 5}$ single-suit hands is included exactly $3$ times in your original count (once with each possible "partner suit"). After excluding these with the correct multiplicity, you have
$$
{4\choose 2}{26\choose 5}-3{4\choose 1}{13\choose 5}=379236
$$
hands containing exactly two suits. As a double-check, you can calculate from the other direction ("inclusion-based"). A hand with two suits has either four cards from one suit and one from the other, or three cards from one suit and two from the other. There are $12$ ways to choose the major and minor suits. For each (ordered) pair of suits, there are ${13\choose 4}{13\choose 1} + {13\choose 3}{13 \choose 2}=31603$ ways to make a hand; the result is $12\times 31603 = 379236$ again.
With the two-suit result in hand, you can finish the problem. The number of single-suit hands, as you argued, is $4\times{13\choose 5}=5148.$ The number of four-suit hands is $4\times 13^3\times{13\choose 2}=685464.$ Therefore, the number of three-suit hands (the only remaining possibility) is ${52\choose 5}-5148-379236-685464=1529112.$ To double-check this, note that a three-suit hand has $(1,2,2,0)$ or $(3,1,1,0)$ cards per suit. There are $12$ ways to choose the "loner" (first) and "excluded" (last) suits. For each (ordered) pair of suits, there are $13\times{13\choose 2}^2 + {13\choose 3}\times 13^2=127426$ ways to make a hand; the result is $12\times 127426 = 1529112$ again. The associated probabilities are:
$$
\begin{eqnarray}
P_1 &=& \frac{5148}{2598960} &=& 0.198\% \\
P_2 &=& \frac{379236}{2598960} &=& 14.592\% \\
P_3 &=& \frac{1529112}{2598960} &=& 58.836\% \\
P_4 &=& \frac{685464}{2598960} &=& 26.375\% \\
\end{eqnarray}
$$
Best Answer
Total no of cases are ${52 \choose 6}$ , i am gonna write ${n \choose r }$ as nCr. (I am assuming that we have to take in consideration that which suit are of the 8 and face cards are of).
Now as we need 6 cards of three 3 different suites, the possible sums can be:- $a)1+1+4\ \ b)1+2+3\ \ c)1+1+1+3\ \ d)1+1+2+2\ \ e)2+2+2$
a)1+1+4 : rather than counting an 8 appears and an face cars appers it's easier to count total - either one of them does not appears
= Total - 8 does not appear - face card does not appear + both 8 and face card does not appear.
(13C1.13C1.13C4 - 12C1.12C1.12C4 - 10C1.10C1.10C4 + 9C1.9C19C4)$\times$(number of ways of selection of suits)
Selection of suits -> 4C2(for selecting 2 suits from 1 cards come in) .2C(1) (from where 4 cards come)=12
So it comes out 465132
Same way b)->2135520 ,c)->757672 ,d)->1820448, e)->570348
Final answer $\frac{465132+2135520+757672+1820448 +570348}{{52 \choose 6}}$
=$\frac{5749120}{20358520}$=0.28239380858