Your second answer is correct.
In how many ways can six boys and four girls be seated around a table if no two girls are adjacent?
Method 1: Suppose one of the boys is Asa. We will use him as a reference point. The remaining five boys can be seated in $5!$ ways as we proceed clockwise around the table from Asa. Seating the six boys creates six spaces in which a girl could be placed, one to the left of each boy. To separate the girls, we must choose four of these six spaces in which to place a girl. The girls can be arranged in the four selected spaces in $4!$ ways as we proceed clockwise around the table from Asa. Hence, the number of admissible seating arrangements is
$$5!\binom{6}{4}4!$$
Method 2: We modify your attempt. Suppose Adrienne is one of the girls. The other girls can be seated in $3!$ ways as we proceed clockwise around the table from Adrienne. This creates four spaces in which to place the boys, one to the left of each girl. Let $x_1, x_2, x_3, x_4$ denote, respectively, the number of boys in the first, second, third, and fourth spaces as we proceed clockwise around the table from Adrienne. Since there are a total of six boys,
$$x_1 + x_2 + x_3 + x_4 = 6$$
Since no two of the girls are adjacent, there must be at least one boy in each of the four spaces. Hence, this is an equation in the positive integers. A particular solution of the equation corresponds to placing three addition signs in the five spaces between successive ones in a row of six ones.
$$1 \square 1 \square 1 \square 1 \square 1 \square 1$$
For instance, placing an addition sign in the first, second, and fourth spaces gives
$$1 + 1 + 1 1 + 1 1$$
which corresponds to the solution $x_1 = 1$, $x_2 = 1$, $x_3 = 2$, $x_4 = 2$. The number of such solutions is the number of ways we can place three addition signs in the five spaces between successive one in a row of six ones, which is
$$\binom{6 - 1}{4 - 1} = \binom{5}{3}$$
One the number of boys in each space has been selected, the boys can be arranged in those spaces in $6!$ ways as we proceed clockwise around the table from Adrienne. Hence, there are
$$3!\binom{5}{3}6!$$
admissible seating arrangements.
Method 3: We correct your approach. Again, suppose Adrienne is one of the girls. The other girls can be seated in $3!$ ways as we proceed clockwise around the table from Adrienne. Choose which four of the six boys will sit to the immediate left of a girl. Those boys can be arranged in $4!$ ways as we proceed clockwise around the table from Adrienne. That leaves two boys to place. There are two possibilities: both boys are placed between the same two girls so that there are three boys between those girls or they are placed between separate pairs of girls so that there are two pairs of girls with two boys between them.
Both of the remaining boys are placed between the same two girls: There are four ways to choose the pair of girls the boys sit between. Both boys must sit to the left of the boy who has already been seated to the immediate left of the first girl we encounter in the pair as we proceed clockwise around the table from Adrienne (starting with Adrienne). There are two ways to choose which of the boys who has not yet been seated sits next to that boy. Hence, there are $\binom{4}{1}2!$ such cases.
The remaining boys are placed between two different pairs of girls: There are $\binom{4}{2}$ ways to choose which two pairs of girls the boys sit between. In each case, the boy must be seated to the left of the boy who has already been seated to the immediate left of the first girl we encounter in the pair as we proceed clockwise around the table from Adrienne (starting with Adrienne). There are two ways to choose which of the boys we will encounter first as we proceed clockwise around the table from Adrienne. There are $\binom{4}{2}2!$ such cases.
Thus, there are
$$3!\binom{6}{4}4!\left[\binom{4}{1}2! + \binom{4}{2}2!\right]$$
admissible seating arrangements.
Your errors:
You forgot to arrange the four boys you placed in the gaps, so you were missing a factor of $4!$. Had you included that factor, your count would have been too large. The reason for this is that you counted the same arrangement more than once. For instance, if Asa, Bradley, and Charles all sit between Adrienne and Bronwyn, your approach would count the same arrangement $3! = 6$ times, corresponding to the $3!$ orders in which the same boys could be placed in the same seats. If Asa and Bradley were to be placed between Adrienne and Bronwyn and Charles and David were to be placed between Bronwyn and Claire, then your approach would count the same arrangement four times, corresponding to the $2!$ orders in which Asa and Bradley could be placed in the same seats and the $2!$ orders in which Charles and David could be placed in the same seats.
Best Answer
Have the $5$ boys stand in a line. This can be done in $5!$ ways. For the moment, add a boy at each end, who will be removed when we're done. Now send the $3$ girls, one at a time, to stand between two boys. The first girl has $6$ choices, the second girl will have $5$ choices, and the third girl will have $4$ choices. This gives a total of
$$5!\cdot6\cdot5\cdot4=14{,}400$$
arrangements. Remove the two extra boys, and have the others sit.
Remark: the fiction of the extra two boys is simply to avoid having to elaborate that each girl can either wedge herself between two boys or else stand next to a boy at one end or the other.