As stated in the comments, the answer you obtained using the second method is correct. We can use the Inclusion-Exclusion Principle to correct your first method.
There are $$\binom{10}{6}$$ distinguishable ways to arrange four blue and six red cards in a row since choosing the positions of the red cards completely determines the arrangement of the cards. From these, we must exclude those arrangements in which there are one or more pairs of adjacent blue cards.
A pair of adjacent blue cards: We have nine objects to arrange: BB, B, B, R, R, R, R, R, R. We choose six of the nine positions for the red cards and one of the three remaining positions for the pair of adjacent blue cards. The other blue cards must fill the remaining two positions. Hence, there are
$$\binom{9}{6}\binom{3}{1}$$
such arrangements.
Two pairs of adjacent blue cards: This can occur in two ways. Either we have an overlapping pair (three consecutive blue cards) or two separate pairs of blue cards.
- Two overlapping pairs: We have eight objects to arrange: BBB, B, R, R, R, R, R, R. We choose six of the eight positions for the red cards and one of the remaining two positions for the three adjacent blue cards. The remaining blue card must be placed in the remaining position. Hence, there are
$$\binom{8}{6}\binom{2}{1}$$
such cases.
- Two separate pairs: We have eight objects to arrange: BB, BB, R, R, R, R, R, R. We choose six of the eight positions for the red cards. The remaining two positions must be filled with the two pairs of adjacent blue cards. Hence, there are
$$\binom{8}{6}$$
such cases.
Three pairs of adjacent blue cards: This can only occur if there are four consecutive blue cards. We have seven objects to arrange: BBBB, R, R, R, R, R, R. We choose six of the seven positions for the six red cards. The four adjacent blue cards must be replaced in the remaining position. Hence, there are
$$\binom{7}{6}$$
such cases.
By the Inclusion-Exclusion Principle, the number of arrangements of four blue and six red cards in which no two of the blue cards are consecutive is
$$\binom{10}{6} - \binom{9}{6}\binom{3}{1} + \binom{8}{6}\binom{2}{1} + \binom{8}{6} - \binom{7}{6}$$
Note: When we subtract those arrangements in which there is a pair of adjacent blue cards, we subtract those arrangements in which there are two pairs of adjacent blue cards twice, once for each way we could designate one of those pairs as the excluded pair. We only want to subtract such cases once, so we must add them back.
When we subtract those arrangements in which there are a pair of adjacent blue cards, we subtract those arrangements in which there are three pairs of adjacent blue cards thrice, once for each way we could designate one of those pairs as the excluded pair. When we then add those cases in which there are two excluded pairs, we add those arrangements in which there are three pairs of adjacent blue cards thrice, once for each of the $\binom{3}{2}$ ways we could designate two of those three pairs as the excluded pairs. Since we have both added and subtracted these arrangements three times, we have not excluded them at all. Therefore, we must subtract the number of arrangements in which there are three excluded pairs.
Best Answer
I think we are to assume bulbs of the same colour are indistinguishable. So we are counting the "words" of length $9$ that have $3$ R, $4$ Y, and $2$ B.
The places for the R's can be chosen in $\binom{9}{3}$ ways. For each of these ways, the places for the Y's can be chosen in $\binom{6}{4}$ ways. Multiply and simplify.
About the books, imagine that that the math books are placed in a box, labelled M. Then we have $9$ objects, the English books and the M. These can be arranged on the shelf in $9!$ ways. For each of these ways, the math books can be taken out of the box and arranged in $4!$ ways, for a total of $9!4!$.
For the probability, divide as you did by $12!$.