Have you considered finding the intersections using an implicit form for the circles, $$\frac{x^2}{r^2} + \frac{y^2}{r^2} + ax + by + c = 0?$$ This representation doesn't have any coefficients that diverge as the circle approaches a straight line. To find intersections, you'll have to solve a quadratic equation whose leading coefficient could be zero or arbitrarily close to it, but the alternative form of the quadratic formula should be able to deal with that robustly.
You'll then have to do some jiggery-pokery to figure out whether the intersection points lie within the arcs. If the arc's bending angle is smaller than $\pi$, a projection onto the line joining the endpoints will suffice.
(Disclaimer: While all of this feels like it should work, I haven't analyzed it in any detail. Also, there could still be a problem when the circle is close to a line and you want the longer arc. But I can't imagine that's a case that would turn up in any practical application.)
Update: For a concrete example, here is the equation for a circular arc passing through the three points $(0,0)$, $(0.5, h)$, and $(1,0)$: $$\kappa^2 x^2 + \kappa^2 y^2 - \kappa^2 x - 2\eta y = 0,$$ where $$\begin{align}\kappa &= \frac{8h}{4h^2 + 1}, \\ \eta &= \frac{8h(4h^2-1)}{(4h^2+1)^2}.\end{align}$$ As you can see, the coefficients remain bounded as $h \to 0$.
Update 2: Wait, that equation becomes trivial if $h = 0$, which is bad. We really want something like $x^2/r + y^2/r + ax + by + c,$ i.e. multiply the previous expression through by $r$. Then for the same example, our equation becomes $$\kappa x^2 + \kappa y^2 - \kappa x - 2\eta' y = 0,$$ where $\eta' = (4h^2-1)/(4h^2+1)$. Here are some explicit values.
$h = 1/2$: $$2 x^2 + 2 y^2 - 2 x = 0,$$ $h = 0.01$: $$0.07997 x^2 + 0.07997 y^2 - 0.07997 x + 1.998 y = 0,$$ $h = 0$: $$2 y = 0.$$
By the way, in this format, the linear terms will always be simply $-2(x_0/r)x$ and $-2(y_0/r)y$, where the center of the circle is at $(x_0,y_0)$. As the center goes to infinity but the endpoints remain fixed, these coefficients remain bounded and nonzero (i.e. not both zero).
Best Answer
Extend your drawing as shown below. Let $O$ be the center of curvature of the arc; extend a line through $O$ parallel to the wall through $P$ so that it meets the perpendicular line through $P$ at a point $Q$. Let the radius of the arc be $R$, and let $OQ = a$ and $PQ = b.$ Let $C$ be the (as yet unknown) center of the desired circle and $r$ the (as yet unknown) radius of that circle.
Then $\triangle OQC$ is a right triangle with right angle at $Q$, legs $a$ and $b + r,$ and hypotenuse $R - r.$ By the Pythagorean Theorem,
$$(R - r)^2 = a^2 + (b + r)^2.$$
Expand both sides: $$R^2 - 2Rr + r^2 = a^2 + b^2 + 2br + r^2.$$
Cancel the $r^2$ on each side, collect the terms in $r$ on the right and the other terms on the left: $$R^2 - a^2 - b^2 = 2(R + b)r.$$
Therefore $$r = \frac{R^2 - a^2 - b^2}{2(R + b)} = \frac{R^2 - (OP)^2}{2(R + b)}.$$