For reference: In figure $T$ and $K$ are points of tangency,
$MT = a$ and $KN = b$; calculate area of region $ABC$.
(Answer:$2\sqrt{ab}(\sqrt a+\sqrt b)^2$)
My progress:
$$S_{ABC} = p \cdot r = \frac{r \cdot (AB+BC+AC)}{2}\\
AC +2R = AB+BC\\
S_{ABC} = AG \cdot GC \qquad \text{(property)} \\
S_{ABC} = (AC+R)R \qquad \text{(property)} \\
OTBQ:~\text{square} \implies TK = R\sqrt2 \\
\ldots ?$$
I'm not able to use segments a and b in the resolution
Best Answer
Here is a geometrical solution without much algebra.
For a right triangle, if you draw a line through the points of tangency of the incircle with the perpendicular sides, it does bisect the arcs of the circumcircle on both sides. In other words, $M$ and $N$ are midpoints of minor arcs $AB$ and $BC$ respectively. At the end of the answer, I have shown a proof.
With that, note that $\triangle BTM \sim \triangle NKB$. That leads to,
$\frac{r}{a} = \frac{b}{r} \implies r = \sqrt{ab}$
As $FM$ is perpendicular bisector of $AB$ and $FN$ is perpendicular bisector of $BC$,
$\frac{AB}{2} = \sqrt{ab} + \frac{a}{\sqrt2}$ and $\frac{BC}{2} = \sqrt{ab} + \frac{b}{\sqrt2}$
As we know $AB$ and $BC$ in terms of $a$ and $b$, we are done, for $S_{\triangle ABC} = \frac 12 \cdot AB \cdot BC$.
Proof of the property that I used in the above answer -
Say $M$ and $N$ are midpoints of the arcs $AB$ and $BC$ and segment $MN$ intersects $AB$ and $BC$ at $T$ and $K$ respectively.
$\angle BIN = 45^\circ + \angle A/2$ So, $\angle KPN = 90^\circ + \angle A/2$
Also, $\angle PNK = \angle C/2$
That leads to $\angle BKM = \angle PKN = 45^\circ$
Also note that $\angle INK = \angle ICK = \angle C / 2$ so $ICNK$ is cyclic and therefore $\angle KIN = \angle KCN = \angle A / 2$
That leads to $\angle IKM = \angle A / 2 + \angle C / 2 = 45^\circ$
$\angle BKI = \angle BKM + \angle IKM = 90^\circ$.
So $K$ must be point of tangency of incircle with side $BC$. Finally since $KI \parallel BT$ and $BT = BK = KI$, $T$ is the point of tangency of incircle with side $AB$.