Excuse the poor drawing.
$\triangle CDE$ is an equilateral triangle inscribed inside a circle, with side length $16$. Let $F$ be the midpoint of $DE$. Points $G$ and $H$ are on the circle so that $\triangle FGH$ is an equilateral. Find the side length of $\triangle FGH$ and express it as $a\sqrt{b}-c$ where $a, b, c$ are positive integers.
My attempt
$$\theta \text{ in the Large } \triangle = \theta \text{ in the Small } \triangle = 60^{\circ}$$
$$(\text{side of the Large } \triangle) \cdot \sin 60^{\circ} + (\text{side of the Small } \triangle )\cdot \sin 60^{\circ} = 2\cdot(\text{radius of circle})$$
Solving for the small side, I got $\dfrac{64}{3}-16$, which doesn't match the required form.
Thanks for any help.
Best Answer
This answer may not be the simplest, but it is straightforward.
I particularized the problem by making one side of the large equilateral triangle the segment between the points $(-8,0)$ and $(8,0)$. Then simple geometry tells us the third vertex is at $(8\sqrt{3},0)$, the circumcenter of the triangle is at $A(0,\dfrac{8\sqrt{3}}3)$, and the radius of the circumcircle is $\dfrac{16\sqrt{3}}3$.
The equation of the circumcircle is then
$$x^2+\left(y-\frac{8\sqrt{3}}3\right)^2=\left(\frac{16\sqrt{3}}3\right)^2$$ $$x^2+\left(y-\frac{8\sqrt{3}}3\right)^2=\frac{256}3$$
The one side of the small equilateral triangle, $\overline{FH}$, is on the line $y=-\sqrt{3}x$. That gives us two equations in two unknowns for the coordinates of point $H$ which is on both the circle and the line. Substitute the expression for $y$ in the linear equation into the quadratic equation and we get a quadratic equation for $x$:
$$x^2+\left(-\sqrt 3x-\frac{8\sqrt 3}3\right)^2=\frac{256}3$$ $$4x^2+16x-64=0$$ $$x^2+4x-16=0$$
Solving this gives us this positive value for $x$:
$$x=2\sqrt 5-2$$
The side of the small equilateral triangle is twice the $x$-coordinate of point $H$, so the side of the triangle is
$$4\sqrt 5-4$$
The final answer to your problem, given the side is $a\sqrt b-c$, is
I checked this answer numerically with Geogebra, the source of my diagram above, and my answer checks.