0 choices for the 1st person.
17 choices for the 2nd person (must exclude 1st and his/her two neighbours)
For 2 of these choices of 2nd person, there is one shared neighbour, so 15 remaining choices. (e.g. if they are numbered 1 to 20 in a circle, 1st person is #1, 2nd is #3, then people 20,1,2,3,4 are excluded).
For the other 15 choices of 2nd person, there are no shared neighbors, so 14 remaining choices.
So if order matters, total is $20 \cdot (2 \cdot 15 + 15 \cdot 14) $
but since order does not matter, divide by $3! = 6$ to account for the permutations in order of the 3 people.
So total = $20 \cdot \frac{2 \cdot 15 + 15 \cdot 14}{6}$
just redid it; does this make any sense?
Best Answer
There are $20$ ways to choose a block of $3$ consecutive people. There are also $20$ ways to choose a pair of adjacent people, and for each of those pairs there are $16$ ways to choose a third person who is not adjacent to either of them. Those are the only sets of $3$ people that we don’t want. There are altogether $\binom{20}3$ possible sets of $3$ people, so the number of acceptable sets is
$$\binom{20}3-20-20\cdot16=\frac{20\cdot19\cdot18}6-20\cdot17=20(57-17)=800\;.$$
Added: More generally, suppose that we have $n$ people at the table, and we want to choose $k$ of them, no two of whom are adjacent. First solve the problem when the $n$ people are in a line. Once the $k$ people are chosen, there will be $k+1$ gaps in which the others can be seated: one before the first chosen person, $k-1$ between adjacent chosen people, and one after the last chosen person. We have to fill these gaps with the $n-k$ people whom we didn’t choose. We must be sure to put at least one person in each of the $k-1$ gaps between adjacent chosen people, so we have only $n-k-(k-1)=n-2k+1$ people left to distribute freely amongst the $k+1$ gaps. This can be done in
$$\binom{(n-2k+1)+(k+1)-1}{(k+1)-1}=\binom{n-k+1}k$$
ways, by a standard stars and bars calculation.
However, this count includes the arrangements that in which the two people at the ends of the line are chosen, and when we wrap the line back around the table, those two people are adjacent. Thus, we don’t want to count those arrangements. There is one of them for each way of choosing $k-2$ people from a line of $n-2$ people with the requirement that no two be adjacent and that neither end person be chosen. This time we’re distributing $n-k$ unchosen people amongst $k-1$ slots with a requirement that each slot get at least one person, and another stars and bars calculation gives the number of such distributions as
$$\binom{n-k-1}{k-2}\;.$$
The final answer is therefore
$$\binom{n-k+1}k-\binom{n-k-1}{k-2}\;,$$
which in this specific problem is
$$\binom{18}3-\binom{16}1=816-16=800\;.$$