Consider the parametrization of the circle $(x_c(\theta), y_c(\theta))=(r\cos \theta,r\sin \theta)$. It's derivative $(x_c'(\theta), y_c'(\theta))=(-r\sin \theta,r\cos \theta)$ gives the direction of the tangent line at $\theta$.
For every $\theta$, this vector is perpendicular to the tangent of the involute:
$$
(x_c'(\theta),y_c'(\theta))\cdot(x'(\theta),y'(\theta)) = (-r\sin\theta)(r\theta \cos\theta) + (r\cos\theta)(r\theta \sin \theta) = 0.
$$
This proves the statement.
Alternatively, it suffices to note that for every $\theta$ the vector $(\cos \theta,\sin \theta)$ is normal to the circle. And since $(x'(\theta),y'(\theta))$ is just a multiple of the normal, the statement follows.
Here's a picture with some more annotations:
I've highlighted, in blue, an arc $c$ of the larger circle and, in red, an arc $d$ of the smaller circle, as well as drawn, in green, an arrow from the center of the smaller circle $B$ to $P$. The part of the expression you understand is essentially the position of $B$, so let's look at how to determine the vector from $B$ to $P$.
The first thing to observe is that the arcs $c$ and $d$ must have the same length, since they are rolling on one another. Let $\alpha$ be the angle $\angle ABP$. The length of $c$ is $\theta a$ and the length of $d$ is $\alpha b$, since arc length is just angle times radius. Thus $\alpha = \frac{a}b\cdot \theta$.
To figure out the vector for the green arrow, however, we really want the angle between $BP$ and a horizontal reference line - the same way that we figured out the vector from $A$ to $B$. Suppose that the angle of $BP$ counterclockwise from horizontal is $\kappa$, the same way that $\theta$ is the angle of $AB$ counterclockwise from horizontal. We can determine $\kappa$ by extending the horizontal line through $A$ and also extending the line $BP$. These extended lines intersect at some point $I$. The triangle $ABI$ has angles $\theta$ and $\alpha$ and $-\kappa$ - the negative sign being since the angle measured at $\angle BIA$ is actually how far clockwise of horizontal the vector $BP$ is from horizontal where we really wanted to know a counterclockwise angle for consistency. Thus, since the sum of the angles of a triangle is $\pi$, we get $$\kappa = -\pi + \alpha + \theta = -\pi + \frac{a}b \theta + \theta = -\pi + \frac{a+b}b \theta$$
This gives that the displacement of the vector $BP$ is
$$\begin{bmatrix}b\cos\left(-\pi+\frac{a+b}b\theta\right)\\b\sin\left(-\pi+\frac{a+b}b\theta\right)\end{bmatrix}$$
which, since $\sin(x+\pi)=-\sin(x)$ and $\cos(x+\pi)=-\cos(x)$,
$$\begin{bmatrix}-b\cos\left(\frac{a+b}b\theta\right)\\-b\sin\left(\frac{a+b}b\theta\right)\end{bmatrix}.$$
Summing this to the center of the smaller circle gives the answer.
You could also get this by imagining that instead of the smaller circle rolling on the larger one, they are both keeping their centers constant and spinning without slipping (like gears). Then, the calculation of $\alpha$ would tell you that the smaller circle is spinning at a ratio of $\frac{a}b$ times the speed of the larger one. In this view, a point on the smaller circle is just tracing the circumference of the smaller circle at a fixed rate - which is easy enough to describe in cartesian coordinates.
If you then imagined that you were an observer standing on and rotating with the larger circle, you would realize that, from your point of view, the smaller circle was revolving around you and moving without slipping - as in the original problem. You could thus also get the answer by starting with the "both circles spin in place" case and shifting reference frames appropriately - essentially by rotating all of space over time. This is definitely not easier algebraically, but it might be helpful conceptually. (Or might not - it's at least a fun way to look at it)
Best Answer
Another way to solve without vectors.
The parametrics have an easy elimination of parameter $t$ which gives the nodal cubic of cartesian equation $$y^2=(x+1)x^2$$ so the derivative is $$2ydy=(x^2+2x(x+1))dx\Rightarrow \frac{dy}{dx}=\frac{3x^2+2x}{2x\sqrt{x+1}}$$ The value of this derivative at $x=0$ has the form $\frac 00$ so applying l'Hôpital Rule we get the form $$\frac{6x+2}{2\sqrt{x+1}+\frac{x}{\sqrt{x+1}}}$$ which have the determined value $1$ when $x=0$ and clearly $-1$ for the other negative value of $y=\pm x\sqrt{x+1}$.
Thus the searched angle is $90^{\circ}$.