The double angle formula for sine is:
$$\sin(2θ)=2\sin(θ)\cos(θ)$$
rather than $\frac12\sin(θ)\cos(θ)$ like you've written.
Hence in the end you have:
$$A=4\left(\frac{θ}{2}-\frac{\sin(2θ)}{4}\right)$$
So apart from this small mistake, the rest of your calculations are fine, and the final (corrected) formula agrees with the formula provided in the Wolfram link you posted, if you substitute $R=r=1$, and use the fact that $d=2\cos(θ)$.
And, as you pointed out, you can make the formula look a bit cleaner by setting $φ=2θ=2\operatorname{arccos\left(\frac{d}{2}\right)}$ to get:
$$A=φ-\sin(φ)$$
If OP’s statement holds, then there exists a unique angle $\measuredangle BAF$ (see the diagram), which does not depend on the radii of the given Sangaku configuration. For brevity, we denote this particular angle by $\alpha$. Our attempt is to find $\alpha$.
Note that the line segments $AF$ and $HC$ are externally tangent to the two $\color{red}{\text{red circles}}$ and the $\color{blue}{\text{blue circle}}$, the radii of which are $r_a$ and $r_b$ respectively. Vertical line segment $UV$ is externally tangent to the leftmost red circle and the blue circle at $S$ and $T$ respectively. The radius of the black circumscribing circle is $r_a$. This circle is internally tangent to the leftmost red circle and the blue circle at $M$ and $B$ respectively.
Two triangles $ADO_a$ and $AEO_b$ are right-angled.
$${\large{\pmb{\therefore}}}\quad O_aA=\dfrac{r_a}{\sin\left(\alpha\right)}\quad\text{and}\quad O_bA=\dfrac{r_b}{\sin\left(\alpha\right)}.$$
We know that, $O_bA=O_aA+O_aC+O_bC$. Therefore, we can express $r_a$ in terms of $r_b$ and $\alpha$ as shown below.
$$O_bA=\dfrac{r_b}{\sin\left(\alpha\right)}=\dfrac{r_a}{\sin\left(\alpha\right)}+r_a+r_b\quad \longrightarrow\quad r_a=\dfrac{1-\sin\left(\alpha\right)}{1+\sin\left(\alpha\right)}r_b\tag{1} $$
Similarly, since we know that, $AB=O_bA+O_bB$, we can express $r_c$ in terms of $r_b$ and $\alpha$.
$$2r_c=AB=\dfrac{r_b}{\sin\left(\alpha\right)}+r_b\quad \longrightarrow\quad r_b=\dfrac{2\sin\left(\alpha\right)}{1+\sin\left(\alpha\right)}r_c \tag{2}$$
While deducing (1) and (2), we only considered spatial data of blue circle, middle red circle, and chord $AF$ with respect to black circle. To incorporate constrains imposed on the leftmost red circle by black circle and semi-chord $HC$, we need to derive the following equation.
Applying Pythagoras theorem to the right-angled triangle $O_cO_aO’_a$, we can write,
$$ (O'_aO_c)^2=(O_aO_c)^2+(O'_aO_a)^2\quad\longrightarrow\quad (r_c-r_a)^2=(2r_b-r_c+r_a)^2+(r_b-r_a)^2.$$
Assuming $r_b\neq 0$, this can be simplified to get,
$$ r_c=\dfrac{(r_b+r_a)^2}{4r_b}+r_b .\tag{3}$$
Now, we are in a position to check whether there exists a unique $\alpha$. To do this, substitute (1) and (2) in (3).
$$ 4\dfrac{1+\sin(\alpha)}{2\sin(\alpha)}r_b^2=\left(r_b+\dfrac{1-\sin(\alpha)}{1+\sin(\alpha)}r_b\right)^2+4r_b^2$$
It is obvious that $r_b$ can be eliminated from the above equation, because $r_b\neq 0$. Further simplification yields the following cubic equation.
$$ \sin^3(\alpha)+\sin^2(\alpha)+ \sin(\alpha) =1\tag{4}$$
It can be shown that the discriminant of the reduced form of (4) is greater than zero, which indicates that (4) has only one real root. With that we have proven that the Sangaku configuration described in OP’s problem statement is possible. To find the exact value of $\sin(\alpha)$, we can use Cardano’s method and Vieta’s formula.
$$\sin(\alpha)=\sqrt[3]{\dfrac{17}{27}+\sqrt{\left(\dfrac{17}{27}\right)^2+\left(\dfrac{2}{9}\right)^3}}+\sqrt[3]{\dfrac{17}{27}-\sqrt{\left(\dfrac{17}{27}\right)^2+\left(\dfrac{2}{9}\right)^3}}-\dfrac{1}{3}$$
$$=\dfrac{1}{3}\left(\sqrt[3]{17+\sqrt{297}} +\sqrt[3]{17-\sqrt{297}}-1\right).\qquad\qquad\qquad\qquad$$
With this we have $\enspace\alpha=32.935120800772722797935101378469^o$.
We substitute this value in (1) and (2) to express $r_b$ and $r_a$ in terms of $r_c$.
$$r_b=0.70440225747791522901900340714848\times r_c,\quad\text{and}$$
$$r_a=0.20821971713793204783928464924803\times r_c\qquad\quad$$
Best Answer
The area changes continuously, you just get a pointy bit at the top of the graph.
Perhaps you meant that the graph isn't differentiable at the top? That's true: if you think about it, the area is going up very fast "just before" the circles are coincident, but just afterwards, the area goes down very quickly. This means that the graph is not differentiable at the point where the circles are coincident. Intuitively, as you approach coincident circles, the derivative approaches adding infinitesimal semicircles, followed by immediately subtracting them once you cross.