Writing $a := |BC|$, $b := |CA|$, $c := |AB| = \sqrt{a^2+b^2}$, and $r = |PE| = |PF|$ (so that $|PD| = 3r$), we have
$$\begin{align}
|\triangle ABC| &= |\triangle ABP| + |\triangle BCP| + |\triangle CAP| \\[4pt]
\implies \qquad \frac{1}{2} |BC||CA| &= \frac{1}{2} \left(\; |AB| |PF| + |BC||PD| + |CA||PE| \;\right) \\[4pt]
\implies \qquad a b &= c r + 3 a r + b r = r ( 3 a + b + c )\\[6pt]
\implies \qquad r &= \frac{ab}{3 a + b + c} = \frac{ab}{3 a + b + \sqrt{a^2+b^2}}
\end{align}$$
To address @DanielV's suggestion of generalizing to higher dimensions, consider a right-corner tetrahedron $OABC$, with right corner at $O$ and edge lengths $a := |OA|$, $b := |OB|$, $c := |OC|$. (Note that I'm changing notation slightly from the above.) Let a sphere with center $P$ and radius $r$ be tangent to the faces around vertex $A$, and let a congruent sphere (tangent to the first) be tangent to the faces around vertex $O$. Then $P$ has distance $r$ from faces $\triangle OAB$, $\triangle OCA$, $\triangle ABC$ (the ones touching $A$), and distance $3r$ from face $\triangle OBC$ (the one opposite $A$).
Here's a poor attempt at a diagram:
(In this case, the altitudes from $P$ are color-coded to match their parallel counterparts through $O$. The black altitude is to face $\triangle ABC$.)
Thus,
$$\begin{align}
|OABC| &= |OABP| + |OBCP| + |OCAP| + |ABCP| \\[4pt]
\implies \qquad \frac{1}{6}a b c &= \frac{1}{3}\left(\; r\;|\triangle OAB| + r \;|\triangle OCA| + r\;|\triangle ABC| + 3r\;|\triangle OBC| \;\right) \\[4pt]
&= \frac{1}{3}r \cdot \frac{1}{2} \left(\; a b + c a + 3 b c + 2\;|\triangle ABC| \;\right) \\[6pt]
\implies \qquad r &= \frac{abc}{3bc + ab + ca + 2\;|\triangle ABC|} \qquad (\star)
\end{align}$$
Fun fact: The Pythagorean Theorem for Right-Corner Tetrahedra says that
$$|\triangle ABC|^2 = |\triangle OBC|^2 + |\triangle OCA|^2 + |\triangle OAB|^2$$
so that we have
$$|\triangle ABC| = \frac{1}{2} \sqrt{\; b^2 c^2 + c^2 a^2 + a^2 b^2 \;}$$
and $(\star)$ becomes
$$r = \frac{abc}{3bc + ab + ca + \sqrt{\; b^2 c^2 + c^2 a^2 + a^2 b^2 \;}}$$
In $4$-dimensional space (where there's an analogous Pythagorean Theorem, as there is in any-dimensional space), we have
$$r = \frac{abcd}{3bcd + acd + abd + abc + \sqrt{\;b^2 c^2 d^2 + a^2 c^2 d^2 + a^2 b^2 d^2 + a^2 b^2 c^2\;}}$$
and so forth.
Incidentally, the matching-notation version of the initial answer is
$$r = \frac{ab}{3b + a + \sqrt{\;b^2 + a^2\;}}$$
From the Cosine Law,
$$AB^2+AC^2-BC^2=2\cdot AB\cdot AC\cdot \cos A \implies AB = 5 \text{ or } 16$$
The area of $\triangle ABC$ can be written as the following two ways: $$S_{ABC}=\frac{1}{2}\cdot AB\cdot AC\cdot \sin A = \frac{1}{2}\cdot(AB+AC+BC)\cdot r$$
where $r$ is the radius of the incircle of $\triangle ABC$. This will give us $$r=\frac{AB\cdot AC\cdot \sin60^\circ}{AB+AC+BC}$$ By plugging in the values of $AB, AC, BC$ and $\sin A$, we can get $r$. Further, because $BP=BQ,\ CQ=CR,\ AR=AP$, and $$BP+AP=AB\\BQ+CQ=BC\\AR+RC=AC$$ we know that $$BQ = \frac{BA+BC-AC}{2}$$Therefore, the following two cases are:
Case 1: $AB=5$, then $r=\frac{7\sqrt{3}}{6}$ and $BQ=\frac{3}{2}$.
Case 2: $AB = 16$, then $r = 3\sqrt{3}$ and $BQ = 7$.
Best Answer
$$( a - r ) + (b - r ) = 2 R $$