$12$ points are arranged around a circle. How many ways can you create two pairs of distinct triangles that overlap

combinatorics

So this question was actually in three parts including a part that asked for non-overlapping triangles which I correctly calculated to be $2772$. Take this away from the total amount of triangles $\frac{12C3\cdot9C3}{2}=9240$ gives the correct answer 6468. However, just for practice how would you calculate this from scratch without using the non-overlapping triangles?

I think my attempt is very close but I'm not getting the answer:

  1. Choose the first triangle first by pinning down and disregarding any two distinct points from the circle effectively dividing it into two segments (major and minor). In this case to make it systematic, choose them 4 points apart (that is the minimum because 3 points apart will not work as they will not overlap)
  2. Then select one point in the minor segment to complete the triangle $**2C1**$
  3. To make the other triangle, select another point in the minor segment to ensure overlapping $**1C1**$
  4. To complete the other triangle, select two more points from the major segment $**8C2**$

Now repeat this process by gradually increasing the gap between the pinned-down points so you have

$$\frac{(2C1\cdot1C1\cdot8C2+3C1\cdot2C1\cdot7C2+4C1\cdot3C1\cdot6C2+5C1\cdot4C1\cdot5C2+6C1\cdot5C1\cdot4C1+7C1\cdot6C1\cdot3C2+8C1\cdot7C1\cdot2C2)}{2}\cdot12 = 5442$$
The answer is 6468

Edit: the triangles are selected from distinct points (no repetition between the two triangles)

Best Answer

Pick any of the $12$ points as a base point and number the points $1$ through $12$ clockwise starting at the base point. There are $\binom{12}6=924$ ways to choose $6$ of them to be the vertices of the two triangles.

Say that we choose points $p_1,\ldots,p_6$, where $p_1<\ldots<p_6$. There are $7$ ways to assign them to two triangles so that the triangles overlap. If we label a point $0$ for one triangle, $1$ for the other, and then list the labels of $p_1$ through $p_6$ in that order, the acceptable possibilities are $001011$, $001101$, $010011$, $010101$, $010110$, $011001$, and $011010$: these are the labelings that are not cyclic permutations of $000111$, since it is precisely the cyclic permutations of $000111$ that correspond to non-overlapping triangles. And

$$7\binom{12}6=7\cdot 924=6468\,$$

as it should.