As mentioned in a comment, I've noticed that my method is inadequate once I saw that putting $R=0$ will lead to no rotation at all of the supposedly rotating frame. The proper way to describe vectors in different frames is, as @nickbros123 mentioned, via expressing the same vector in different coordinate bases. Once this is done correctly, it seems that the derivation is quite simple:
Let $\mathbf{b_1},\mathbf{b_2},\mathbf{b_3}$ be any orthonormal basis of the non rotating frame $O$ and $\widetilde{\mathbf{b_1}},\widetilde{\mathbf{b_2}},\widetilde{\mathbf{b_3}}$ those of the rotating frame $\widetilde{O}$, that's rotating with angular velocity $\mathbf{\Omega} = (0,0,\omega)$. So we have that:
\begin{align}
\widetilde{\mathbf{b_1}} &= \cos{\omega t}\mathbf{b_1} + \sin{\omega t}\mathbf{b_2} \\
\widetilde{\mathbf{b_2}} &= -\sin{\omega t}\mathbf{b_1} + \cos{\omega t}\mathbf{b_2} \\
\widetilde{\mathbf{b_3}} &= \mathbf{b_3}
\end{align}
Now the statement that for any vector $\mathbf{f}$ it must hold that $\left(\mathbf{f}\right)_O = \left(\mathbf{f}\right)_{\widetilde{O}}$ is simply that the vector is the same regardless of the basis it is expressed by, that is, if it has components $(f_x,f_y,f_z)$ with respect to $O$ and components $(\tilde{f_x},\tilde{f_y},\tilde{f_z})$ with respect to $\widetilde{O}$ then it must hold that:
$$
f_x\mathbf{b_1}+f_y\mathbf{b_2}+f_z\mathbf{b_3} = \tilde{f_x}\widetilde{\mathbf{b_1}}+\tilde{f_y}\widetilde{\mathbf{b_2}}+\tilde{f_z}\widetilde{\mathbf{b_3}}
$$
Differentiating the left hand side simply yields $\left(\frac{d\mathbf{f}}{dt}\right)_O = (\dot{f_x},\dot{f_y},\dot{f_z})$ and there are no additional terms in that basis simply because by assumption $\dot{\mathbf{b_i}} = 0$ for $i=1,2,3$. Differentiating also the right hand side, we get:
\begin{align}
\left(\frac{d\mathbf{f}}{dt}\right)_O &= \dot{\tilde{f_x}}\widetilde{\mathbf{b_1}}+\dot{\tilde{f_y}}\widetilde{\mathbf{b_2}}+\dot{\tilde{f_z}}\widetilde{\mathbf{b_3}}+\tilde{f_x}\dot{\widetilde{\mathbf{b_1}}}+\tilde{f_y}\dot{\widetilde{\mathbf{b_2}}}+\tilde{f_z}\dot{\widetilde{\mathbf{b_3}}} \\ &= \left(\frac{d\mathbf{f}}{dt}\right)_\widetilde{O} + \tilde{f_x}\dot{\widetilde{\mathbf{b_1}}},+\tilde{f_y}\dot{\widetilde{\mathbf{b_2}}}+\tilde{f_z}\dot{\widetilde{\mathbf{b_3}}}
\end{align}
Now since we note that:
$$\dot{\widetilde{\mathbf{b_1}}} = -\omega\sin{\omega t}\mathbf{b_1}+\omega\cos{\omega t}\mathbf{b_2} = \omega\widetilde{\mathbf{b_2}} \\
\dot{\widetilde{\mathbf{b_2}}} = -\omega\cos{\omega t}\mathbf{b_1}-\omega\sin{\omega t}\mathbf{b_2} = -\omega\widetilde{\mathbf{b_1}} \\
\dot{\widetilde{\mathbf{b_3}}} = 0$$
We obtain:
\begin{align}
\left(\frac{d\mathbf{f}}{dt}\right)_O &= \left(\frac{d\mathbf{f}}{dt}\right)_\widetilde{O} + \tilde{f_x}\dot{\widetilde{\mathbf{b_1}}},+\tilde{f_y}\dot{\widetilde{\mathbf{b_2}}}+\tilde{f_z}\dot{\widetilde{\mathbf{b_3}}} \\ &=
\left(\frac{d\mathbf{f}}{dt}\right)_\widetilde{O} + \omega\tilde{f_x}\widetilde{\mathbf{b_2}} -\omega\tilde{f_y}\widetilde{\mathbf{b_1}} \\ &=
\left(\frac{d\mathbf{f}}{dt}\right)_\widetilde{O} + \left(\mathbf{\Omega} \times \mathbf{f}\right)_{\widetilde{O}}
\end{align}
Which appears to be the correct result.
Additional observations and useful resources
Despite the fact that the above argument assume a very special looking form for $\mathbf{\Omega}$ it can in fact be easily generalized, by considering that the orthonormal basis which is our starting point $\mathbf{b_1},\mathbf{b_2},\mathbf{b_3}$ can be related to any other fixed orthonormal basis by a fixed rotation. With respect to this other basis then, $\mathbf{\Omega}$ will no longer have this special form, and yet such a fixed rotation clearly leaves the rotated basis vectors as time independent, and hence does not affect the rest of the derivation.
Since writing this, I have found two very good posts that are very much related and worth reading. The first one is a derivation of the centrifugal and Coriolis force terms that appear in a rotating reference frame, which basically derives the same theorem (without naming it, which is why it took me a long time to find!) in a slightly different way. The second one is a truly beautiful mathematical treatment that much more generally derives the existence of all the known fictitious forces that arise in a non inertial frame of reference, the kinematic transport theorem can also be derived by applying the same techniques used there.
Best Answer
Here are some errors in your setup.
Here's how I would have done this calculation. I'm not 100% sure what you meant to do for the position vector, but let's say the particle is moving in a circle of radius $r$ (an extreme example where in fact the magnitude of the displacement doesn't change and the only thing that needs to be differentiated is the unit vector $\hat{\mathbf{n}}(t)$)
\begin{eqnarray} \mathbf{r} &=& r \cos \omega t \hat{\mathbf{x}} + r \sin \omega t \hat{\mathbf{y}} = r \hat{\mathbf{n}}_1(t) \\ \frac{d\mathbf{r}}{dt} &=& -\omega r \sin \omega t \hat{\mathbf{x}} + \omega r \cos \omega t \hat{\mathbf{y}} = \omega r \hat{\mathbf{n}}_2(t) \\ \frac{d^2\mathbf{r}}{dt^2} &=& - \omega^2 r \cos \omega t \hat{\mathbf{x}} - \omega^2 r \sin \omega t \hat{\mathbf{y}} = - \omega^2 \mathbf{r} = - \omega^2 r \hat{\mathbf{n}}_1(t) \end{eqnarray} where $\hat{\mathbf{n}}_1(t) = \cos (\omega t) \hat{\mathbf{x}} + \sin (\omega t) \hat{\mathbf{y}}$ and $\hat{\mathbf{n}}_2(t) = - \sin(\omega t) \hat{\mathbf{x}} + \cos(\omega t)\hat{\mathbf{y}}$. When doing the derivatives, I chose to split up the vector $\mathbf{r}$ into Cartesian components with unit vectors $\hat{\mathbf{x}}$ and $\hat{\mathbf{y}}$ (which are constant in time), rather than writing the vector as a magnitude times a direction, so that I wouldn't have to worry about differentiating the unit vector directly.
The magnitude of each of these vectors is manifestly positive \begin{eqnarray} |\mathbf{r}| &=& r \\ \left|\frac{d \mathbf{r}}{dt}\right| &=& \omega r \\ \left|\frac{d^2 \mathbf{r}}{dt^2}\right| &=& \omega^2 r \end{eqnarray} If you do things carefully like this, you will always find the magnitude of each vector is positive, for any motion $\mathbf{r}(t)$.