[Edit:I had a couple of links to the original probelm but they have gone the way of all things.]
I have been attempting to solve this problem:
Given three concentric circles of radii 1, 2, and 3, respectively, find the maximum area of a triangle that has one vertex on each of the three circles.
Here is a partial solution (not my own) which I have edited it a little for clarity. Note that $A=1$, $B=2$ and $C=3$:
Let radii A,B, and C be at angles a,b, and c respectively. Position radius A on the positive x-axis at angle $a=0$ (no loss in generality). From the equation for triangle area
(1) area = $\frac12 BC \sin(b-c) + \frac12 CA \sin(c) + \frac12 AB \sin (2\pi -b)$.
Take the total partial of
area w.r.t $b$ and $c$ and set equal to $0$. This gives
(2) $C \cos c = B \cos b$.
Also, from condition (2) extended radii are perpendicular to the triangle side. Next,
the value of angle $b$ is determined. A little fancy geometry shows that $b$ is $225^o$ from
$A$ ($-45^o$ in the third quadrant). From (2) angle $c$ is obtained.
I am happy with the expression for the triangle's area, and also with the differentiation and derivation of $C\cos c = B\cos b$.
But I don't see why the extended radii are perpendicular to the triangle's sides, which makes the centre of the concentric circles the orthocentre of the triangle. And I'm also not seeing the "fancy geometry" that gives the angle $b$, nor, indeed, why angle $b$ is constant.
Could someone please explain what's happening here?
Best Answer
Suppose that the center of the circles did not lie on one of the altitudes of $\triangle ABC$ (as shown in gray). Then by leaving the base of that altitude alone and moving that altitude to the center, we increase the altitude, and thereby the area of the triangle (as shown in black).
$\hspace{3.5cm}$
Therefore, the orthocenter of the triangle must coincide with the center of the circles.
Consider the following diagram, in which $O$ is both the orthocenter of $\triangle ABC$ and the center of the circles:
$\hspace{3.5cm}$
$\triangle AOF$ is similar to $\triangle COD$, thus, $|\overline{OC}||\overline{OF}|=|\overline{OA}||\overline{OD}|$. Furthermore, $\triangle AOE$ is similar to $\triangle BOD$, thus, $|\overline{OA}||\overline{OD}|=|\overline{OB}||\overline{OE}|$. Therefore, define $$ p=|\overline{OC}||\overline{OF}|=|\overline{OA}||\overline{OD}|=|\overline{OB}||\overline{OE}|\tag{1} $$ and $$ a=|\overline{OA}|\quad b=|\overline{OB}|\quad c=|\overline{OC}|\tag{2} $$ With these definitions, we get $$ |\overline{OD}|=p/a\quad|\overline{OE}|=p/b\quad|\overline{OF}|=p/c\tag{3} $$ Note that $$ \frac{|\triangle AOB|}{|\triangle ABC|}=\frac{|\overline{OF}|}{|\overline{OF}|+|\overline{OC}|}=\frac{p}{p+c^2}\tag{4} $$ $$ \frac{|\triangle BOC|}{|\triangle ABC|}=\frac{|\overline{OD}|}{|\overline{OD}|+|\overline{OA}|}=\frac{p}{p+a^2}\tag{5} $$ $$ \frac{|\triangle COA|}{|\triangle ABC|}=\frac{|\overline{OE}|}{|\overline{OE}|+|\overline{OB}|}=\frac{p}{p+b^2}\tag{6} $$ and because $|\triangle AOB|+|\triangle BOC|+|\triangle COA|=|\triangle ABC|$, $(4)-(6)$ yield $$ \frac{p}{p+a^2}+\frac{p}{p+b^2}+\frac{p}{p+c^2}=1\tag{7} $$ Using $a=1$, $b=2$, and $c=3$, we can solve $(7)$ to get $p=1.458757077431284$.
Looking at the area of $\triangle AOB$ in two different ways, we get $$ \begin{align} 4|\triangle AOB|^2=|\overline{AB}|^2|\overline{OF}|^2&=(a^2+b^2-2ab\cos(\angle AOB))(p/c)^2\\ &=a^2b^2\sin^2(\angle AOB)\\ &=a^2b^2(1-\cos^2(\angle AOB))\tag{8} \end{align} $$ Solving $(8)$ for $\cos(\angle AOB)$ using the quadratic formula, and similarly for the other angles, yields $$ \cos(\angle AOB)=\frac{p^2-\sqrt{(p^2-c^2a^2)(p^2-b^2c^2)}}{abc^2}\tag{9} $$ $$ \cos(\angle BOC)=\frac{p^2-\sqrt{(p^2-a^2b^2)(p^2-c^2a^2)}}{a^2bc}\tag{10} $$ $$ \cos(\angle COA)=\frac{p^2-\sqrt{(p^2-a^2b^2)(p^2-b^2c^2)}}{ab^2c}\tag{11} $$ Using the equations above, we get
$$\angle BOC=104.071123766006501^\circ$$ $$|\triangle BOC|=2.909984011512956$$
$$\angle COA=119.094556197774592^\circ$$ $$|\triangle COA|=1.310727640381874$$
$$\angle AOB=136.834320036218908^\circ$$ $$|\triangle AOB|=0.684110332666474$$ Therefore, we get $$|\triangle ABC|=4.904821984561304$$