Just as @Riverboat suggested.
Consider $\triangle{C_1C_2C_3}$
Clearly, $C_1C_2=5$
Let the radius of circle with centre $C_3$ be $r$
So, $C_1C_3=r+3$ and $C_2C_3=r+2$
With the base, $b=C_1C_2=5$ and height, $h=5-r$, we get
$$[\triangle{C_1C_2C_3}]=\frac{1}{2} \cdot 5 \cdot (5-r)\tag{1}$$
Using Heron's Formula, where
$$a=C_1C_2=5, b=C_1C_3=r+3 \text{ and }c=C_2C_3=r+2$$
$$s=\frac{a+b+c}{2}=r+5$$
We get
$$[\triangle{C_1C_2C_3}]=\sqrt{(5+r)\cdot 2 \cdot 3 \cdot r}\tag{2}$$
Using $(1)$ and $(2)$, we get
$$6(r^2+5r)=\frac{25}{4} \cdot (5-r)^2 \Longleftrightarrow r^2-370r+625=0$$
Solving, we get $r=5(37\pm8\sqrt{21})$
But since $C_3$ touches $C$ internally, $r \leq 5$
Thus, $\color{red}{r=5(37-8\sqrt{21})}$
An alternative could be Descartes' Circle Theorem
Using this, we get
$$(k_1+k_2+k_3+k_4)^2=2(k_1^2+k_2^2+k_3^2+k_4^2)$$
Here, $k_i$ represents the curvature and is given by $$k_i=\pm \dfrac{1}{r_i}$$ where
- $+$ sign is given to a circle externally tangent to other circles.
- $-$ sign is given to a circle internally tangent to other circles.
So, $$k_1=-\frac{1}{5}, k_2=\frac{1}{3} \text{ and } k_3=\frac{1}{2}$$
Solving the quadratic in $k_4$, we get two values of which the positive one is the required curvature. Let this value be $m$.
Thus, the radius of the circle is $r_4=\dfrac{1}{m}$
Radius is exactly $\dfrac{72}{23}$, pretty neat. See the figure below, where dotted lines from the center of the circumcribing circle passes through the midpoints of the sides of the triangle.
Let : $B$ be the origin $(0,0)$, and the center of circle of unknown radius $r$ be $(x,y)$. Then we solve the following three equations to find $r$
$\dfrac{3}{2}+\sqrt{x^2+(\dfrac{3}{2}-y)^2}=r$
$2+\sqrt{(2-x)^2+y^2}=r$
$\dfrac{5}{2}+\sqrt{(2-x)^2+(\dfrac{3}{2}-y)^2}=r$
so that $(r, x, y)= \left(\dfrac{72}{23}, \dfrac{36}{23}, \dfrac{24}{23}\right)$
Note: The midpoints of the sides $(0,\dfrac{3}{2})$, $(2,0)$, and $(2,\dfrac{3}{2})$ are centers of circles of radii $\dfrac{3}{2}, 2, \dfrac{5}{2}$.
Given any two tangent circles, their centres and point of tangency are colinear.
Bonus: If right triangle : $\triangle ABC$ has rational sides, then $(r,x,y)$ are also all rationals.
Best Answer
In general, the radius $r$ of a circle inscribed by any two touching circles of radii $a$ & $b$ and their common tangent, is given by Generalized Formula as follows $$\boxed{r=\frac{ab}{(\sqrt a+\sqrt b)^2}}$$ Now, substituting the values of radii $a=2$ of blue circle and $b=1$ of green circle, the radius $r$ of red circle is calculated as follows $$r=\frac{2\cdot1}{(\sqrt 2+\sqrt 1)^2}$$ $$=\frac{2}{3+2\sqrt2}$$ $$=\frac{2(3-2\sqrt2)}{(3+2\sqrt2)(3-2\sqrt2)}$$ $$=2(3-2\sqrt2)$$