From the comments, it seems that you are allowing only hands of the $3$-$2$ type, commonly called Full House, and hands of the $2$-$2$-$1$ type, commonly called Two Pairs.
Full House: The kind we have $3$ of can be chosen in $\binom{13}{1}$ ways. For each such way, the actual $3$ cards can be chosen in $\binom{4}{3}$ ways. for every way of doing these things, there are $\binom{12}{1}$ ways of choosing the kind we have $2$ of, and then $\binom{4}{2}$ ways to choose the $2$ cards, for a total of $\binom{13}{1}\binom{4}{3}\binom{12}{1}\binom{4}{2}$.
Two Pairs: This one is tricky, it is all too easy to double count. The two kinds we have $2$ each of can be chosen in $\binom{13}{2}$ ways. For each of thse ways, the $2$ cards of the higher-ranking kind can be chosen in $\binom{4}{2}$ ways, and for each of these ways the cards of the other kind can be chosen in $\binom{4}{2}$ ways. Finally, after all this has been done, the "junk" card can be chosen in $\binom{44}{1}$ ways, for a total of $\binom{13}{2}\binom{4}{2}\binom{4}{2}\binom{44}{1}$.
Finally, add the two numbers obtained above.
Remark: Your calculation does multiple counting in various ways. For example, consider the hand $\spadesuit$ K, $\heartsuit$ K, $\diamondsuit$ $7$, $\clubsuit$ $7$, $\heartsuit$ Q. This has been double-counted, for you have counted as different the choice $\diamondsuit$ $7$, $\clubsuit$ $7$, $\spadesuit$ K, $\heartsuit$ K, $\heartsuit$ Q. This is because your first choice of kind, among the $\binom{13}{1}$ could have been K, and your second choice among the $\binom{12}{1}$ could have been $7$, or the other way around. But these give the same hand,
Counting in a way that avoids multiple counting can be initially quite tricky. We naturally think in terms of doing things sequentially, "first" then "second." we have to be careful to make sure that different choices of "first" and "second" do not yield the same final hand.
Clearly no full house can have a King. Other than the King, the cards fall into two types:
$A$: Cards with $4$ suits: there are ten of these.
$B$: Cards with $3$ suits: there are two of these (namely $2,3$).
We consider the various types of full houses (here, for instance type $(A,B)$ means that the three of a kind is of type $A$ and the pair is of type $B$).
Type $(A,A)$. There are $10$ ways to choose the rank for the triple, then $\binom 43$ ways to choose the triple. Then there are $9$ ways to choose the rank for the pair and $\binom 42$ ways to choose the pair. Thus $$10\times \binom 43\times 9 \times \binom 42=2160$$
Type $(A,B)$. There are $10$ ways to choose the rank for the triple, then $\binom 43$ ways to choose the triple. Then there are $2$ ways to choose the rank for the pair and $\binom 32$ ways to choose the pair. Thus $$10\times \binom 43\times 2 \times \binom 32=240$$
Type $(B,B)$. There are $2$ ways to choose the rank for the triple, then $\binom 33$ ways to choose the triple. Then there is $1$ way to choose the rank for the pair and $\binom 32$ ways to choose the pair. Thus $$2\times \binom 33\times 1 \times \binom 32=6$$
Type $(B,A)$.There are $2$ ways to choose the rank for the triple, then $\binom33$ ways to choose the triple. Then there are $10$ ways to choose the rank for the pair and $\binom 42$ ways to chpose the pair. Thus $$2\times \binom 33\times 10 \times \binom 42=120$$
Finally we sum to see that there are $2526$ possible full houses. As there are $\binom {47}5$ possible hands the answer is $$\boxed {\frac {2526}{\binom {47}5}=.001647}$$
To complete the problem, note that removing any card shrinks the denominator in the the same way (doesn't matter which you remove). However, if you remove the $\diamondsuit K$ then the numerator does not change at all, since no full house can have a King using this deck. As any other removal shrinks the numerator, we see that this is the best choice.
'
Best Answer
You are correct that your initial logic under-counts, but the correction of $5!$ is too high. In fact the correct factor to adjust by is $10.$
Before incorrectly multiplying by $5!$, you correctly compute the probability of getting the pattern (xxx)(yy) (hopefully it's clear what I mean by that). Of course this is not the only pattern a full house can come in. It can also be like (yy)(xxx), or (y)(xxx)(y), etc. The way to get the answer would be to compute the probability for all these patterns that constitute a full house, and since they're the mutually exclusive ways to get a full house, just add them up to get the probability of a full house.
Of course the probability is the same for all the patterns (though you may want to talk yourself through a weird one like (x)(y)(x)(y)(x) to make sure it is obvious why it must be the same as for your choice.) So it's just a matter of counting all the patterns. Well, there are five spots and you need to choose three to be x so that's ${5\choose 3} = 10.$
In multiplying by $5!$ you forgot that you had already taken into account the permutations of the cards that only interchange the x's and y's amongst themselves. After all, $(1)(3/51)(2/50)$ is the correct probability of drawing three of a kind in a three card hand... you wouldn't multiply that by $3!.$