Look at the following figure carefully,
As the triangle is equilateral ($AC$ is the angle bisector). So, $\angle ACD = 30^{\circ}$
$$\tan 30^{\circ} = \frac{AD}{DC} = 2AD\ (\because DC = 1/2) $$
$$\therefore AD = \frac{1}{2\sqrt{3}}$$
This is the radius of the bigger circle, let its area be $A_1$
$$\therefore A_1 = \frac{\pi}{12}$$
To calculate the radius of the next smaller circle (let it be $x$), please note that
$$AC = \frac{1}{\sqrt{3}}$$
$$AB =\frac{1}{2\sqrt{3}} +x$$
$$\therefore BC = AC - AB =\frac{1}{2\sqrt{3}} -x $$
Note that triangles $ADC$ and $BCE$ are similar.
$$\therefore \frac{AD}{AC} = \frac{BE}{BC}$$
$$\frac{1}{2\sqrt{3}} \times \sqrt{3} = x \times \left( \frac{2\sqrt{3}}{1-2\sqrt{3}x} \right)$$
$$\therefore x = \frac{1}{6\sqrt{3}}$$
Similarly we can find the radii of the next circles. They would be $\frac{1}{18\sqrt{3}}$,
$\frac{1}{54\sqrt{3}}, ...$
Now, the main answer,
The sequence $\frac{1}{6\sqrt{3}},\frac{1}{18\sqrt{3}}, \frac{1}{54\sqrt{3}}, ... $
can be generally written as $\frac{1}{6\sqrt{3}(3)^{n-1}}$
Total area of these circles,
$$T = \frac{\pi}{12} + 3\sum_{n=1}^{\infty} \pi {\left(\frac{1}{6\sqrt{3}(3)^{n-1}} \right)}^2 $$
Notice that,
$$\sum_{n=1}^{\infty} \pi {\left(\frac{1}{6\sqrt{3}(3)^{n-1}}\right)}^2 = \sum_{n=1}^{\infty} \pi {\left( \frac{1}{108}\right)}{\left(3^{-(n-1)}\right)}^2$$
=$$\sum_{n=1}^{\infty} \pi {\left( \frac{1}{108}\right)}{\left(3^{-2(n-1)}\right)}$$
=$$\sum_{n=1}^{\infty} \pi {\left( \frac{1}{108}\right)}{\left(\frac{1}{9}\right)}^{n-1}$$
This is a GP with $a = \frac{\pi}{108}$ and $r = \frac{1}{9}$
For infinite terms, the sum of this GP = $\frac{a}{1-r} = \frac{\pi}{96}$
Now, finally,
$$T = \frac{\pi}{12} + 3 \times \frac{\pi}{96} = \frac{11\pi}{96}$$
$2$ circles touch externally $\iff$ sum of their radii $=$ distance between their centers. Taking unit length to be $1$cm in Cartesian coordinate system we can express the given circles in the following way $$x^2+y^2=11^2\cdots(1)\\x^2+(y-20)^2=9^2\cdots(2)\\(x-a)^2+(y-b)^2=5^2\cdots(3)$$ As $(3)$ touches $(1),(2)$ externally,
$$a^2+b^2=(5+11)^2,a^2+(b-20)^2=(5+9)^2\Rightarrow a={3\sqrt{55}\over 2},b={23\over 2}$$ The largest circle, let's call it $(4)$, inside the region bounded by $(1),(2),(3)$, it touches $(1),(2),(3)$. Suppose $(4)$ is given by $$(x-c)^2+(y-d)^2=r^2\\\therefore c^2+d^2=(r+11)^2\cdots(5)\\c^2+(d-20)^2=(r+9)^2\cdots(6)\\(c-a)^2+(d-b)^2=(r+5)^2\cdots(7)\\
(6)-(5)\equiv d={r\over 10}+11\cdots(8)\\(5)-(8)^2\equiv c^2=99{r\over 10}\left({r\over 10}+2\right)\cdots(9)$$ Let ${r\over 10}=q$.
$$(7)-(5)\equiv ac+bd=60q+176\Rightarrow c={97q+99\over 2a}\cdots(10)$$
$(10)^2-(9)$ gives us a quadratic equation in $q$ solving which we get $$r={495(-199+30\sqrt{55})\over 9899}\approx1.17$$
Best Answer
See the figure below. One unit on the paper is six units in your problem. $AB=48,AC=40,BC=40$. Circle $D$ has radius $12$ as you say. $HI$ is tangent to both circles and parallel to $AB$, so $ABC$ is similar to the small triangle cut off by $HI$. $EC=32$ by Pythagoras, $EG=24$ from the circle, so $CG=8$ and the small triangle is $\frac 14$ the size of the large one. That says the radius of the small circle is $\frac 14 \cdot 12=3$