Your question is very interesting. Let me give my idea.
Firstly, we will compute $IG$. Note that we have $$\vec{GA} + \vec{GB} + \vec{GC} = \vec{0}.$$ Then, $$3\vec{IG} = \vec{IA} + \vec{IB} + \vec{IC}.$$
$$9IG^2 = \sum_{\circlearrowleft} IA^2 + 2\sum_{\circlearrowleft}\vec{IA}\vec{IB} $$
Recall that $IA^2 = r^2 + (s-a)^2$ (I don't go to this detail, but you can see it easily), and $2\vec{IA}\vec{IB} = IA^2 + IB^2 - AB^2$.
Then, one has: \begin{eqnarray}9IG^2 &=& 3\sum_{\circlearrowleft} IA^2 - \sum_{\circlearrowleft} a^2 = 9r^2 + 3[(s-a)^2 + (s-b)^2 + (s-c)^2] - (a^2+b^2+c^2) \\ &=& 9r^2 + 5s^2 - 4(ab+bc+ca). \ \ \ (1)\end{eqnarray}
Now, use the Heron's formula, we have $$S = sr = \sqrt{s(s-a)(s-b)(s-c)}$$.
So, $$sr^2 = s^3 - s^2(a+b+c) + s(ab+bc+ca) - abc = - s^3 + s(ab+bc+ca) -abc.$$
Note that $abc= 4RS = 4Rrs$. We get $$sr^2 = -s^3 + s(ab+bc+ca) -4Rrs$$ or $$ab+bc+ca = r^2+s^2+4Rr\ \ \ \ (2)$$
Substitute (2) to (1), we get $$9IG^2 = 9r^2 + 5s^2 - 4(r^2 + s^2 + 4Rr) = 5r^2 + s^2 - 16Rr.$$
EDIT: Let me compute the $IH$. I use two lemmas.
Lemma 1 One has $$a\vec{IA} + b\vec{IB} + c\vec{IC} = \vec{0}.$$
Lemma 2 One has $$HA^2 = 4R^2 - a^2; HB^2 = 4R^2 - b^2; HC^2 = 4R^2-c^2.$$
I don't give the detail prove of two above lemmas, but you can see it like this:
- lemma 1: the bisector of angle A divides the side $BC$ by ratio $b:c$, and so on with two other bisectors.
- lemma 2: it is easy to use the Euler line, which implies that $HA$ is two times of the distance from the circumcenter to the side $BC$.
Using lemma 1, we have $$2s \vec{HI} = a\vec{HA} + b\vec{HB} + c\vec{HC}.$$
Then, \begin{eqnarray}4s^2 HI^2 &=& \sum_{\circlearrowleft} a^2HA^2 + 2\sum_{\circlearrowleft} ab \vec{HA}\vec{HB}\\
&=& \sum_{\circlearrowleft} a^2(4R^2 - a^2) + \sum_{\circlearrowleft} ab (HA^2 + HB^2 -AB^2)\ \ \ \ \ \mbox{(using the lemma 2)}\\
&=& 4R^2(a^2+b^2+c^2) - (a^4+b^4+c^4) + \sum_{\circlearrowleft} ab (8R^2 - a^2-b^2-c^2) \\
&=& 4R^2(a^2+b^2+c^2 + 2ab +2bc +2ca) - (a^4+b^4+c^4) - (ab+bc+ca)(a^2+b^2+c^2) \\
&=& 16R^2s^2 - (a^4+b^4+c^4) - (ab+bc+ca)(a^2+b^2+c^2).\end{eqnarray}
I leave the part $(a^4+b^4+c^4) + (ab+bc+ca)(a^2+b^2+c^2)$ for you, because of the simple but long computation.
I give the answer, $$4s^2IH^2 = 16R^2s^2 + 16Rrs^2 + 12r^2s^2 - 4s^4.$$
Then, you get the equality.
Consider how a triangle might contain its circumcenter (the center of its own circumcircle).
Let $\triangle ABC$ be a triangle whose circumcenter is inside the triangle.
Consider the side $AB,$ which is also a chord of the circumcircle.
The chord $AB$ divides the circumcircle into two arcs. The shorter arc is on the side of $AB$ opposite from the center of the circle, and the longer arc is on the same side as the center.
The vertex $C$ also has to be on the same side of $AB$ as the center of the circle in order for the center to be inside the triangle. (The entire inside of the triangle is on the same side of $AB$ as $C$ is.)
And since $C$ lies on the circle also, the angle $\angle ACB$ is an angle inscribed in the circle, and the arc of the circle inside that angle is the arc on the opposite side of $AB$ from $C,$ that is, the shorter arc between $A$ and $B,$ which is less than $180$ degrees, and so $\angle ACB$ is less than $90$ degrees.
That is, the angle at $C$ is acute.
But that's not just a special property of the vertex $C.$ We can consider the side $BC$ and conclude that $A$ is acute, or consider the side $AC$ and conclude that $B$ is acute. In fact all three conclusions are necessarily true.
The circumcenter of a triangle can be inside the triangle only if all three angles of the triangle are acute.
If one angle of a triangle is a right angle, the triangle is a right triangle and its circumcenter lies on the hypotenuse.
This is the only way for the circumcenter to be exactly on a side of the triangle,
because if it is exactly on a side then that side is a diameter and the third angle is $90$ degrees.
The only other possibility--center not inside, center not exactly on a side--is for the center to be outside the triangle. And that's what must happen if one angle of the triangle is obtuse, because that makes it impossible for either of the other two cases to occur.
If any angle of a triangle is obtuse, the circumcenter is outside the triangle.
If the base angle of an isosceles triangle is less than $45$ degrees, then the apex angle is greater than $90$ degrees.
That is, the apex angle is obtuse.
Therefore the circumcenter is outside the triangle.
To understand the fact about the perpendicular lines,
notice that an isoceles triangle has mirror symmetry around the line $AM$ through the apex $A$ and the midpoint $M$ of side $BC.$
The line $AM$ is perpendicular to side $BC,$ and you can flip the triangle over the line $AM$ and get a triangle occupying the exact same place in the plane.
(The locations of $B$ and $C$ will have been swapped, but both points will still be occupied by a vertex of the triangle.)
Because the triangle is mirror symmetric around $AM,$ its incircle and circumcircle also must be mirror symmetric around $AM.$ Otherwise you could flip the entire figure (triangle and circles) over the line $AM$ and get a new incircle and/or new circumcircle the the same triangle.
In order to have this symmetry, the centers of both circles must be on the line $AM,$ which is perpendicular to $BC.$
In fact, the apex vertex $A,$ the midpoint of the opposite side, the incenter and the circumcenter are all on the same line perpendicular to $BC.$
Best Answer
The "$\alpha$" triangle on the left of Figure 3 is an "$\alpha$" triangle from Figure 1b, but scaled by $yz$; so, instead of side-lengths $r$, $x$, $w \;(=\sqrt{r^2+x^2})$, it has side-lengths $r\cdot yz$, $x \cdot yz$, $w \cdot yz$.
Likewise, the "$\alpha$" triangle in the upper-right of Figure 3 is scaled by $rz$ to have side-lengths $x\cdot rz$, $r \cdot rz$, $w\cdot rz$.
This gives the "$\beta$" triangle a scale factor of $wz$ (leg-lengths $y\cdot wz$ and $r\cdot wz$); and, from comparing the top of the rectangle to the bottom, we have that the "$\gamma$" triangle has scale factor $r(x+y)$ (leg-lengths $z\cdot r(x+y)$ and $r\cdot r(x+y)$).