I think your proof has the right idea, but I would advise against using indefinite integrals in a formal proof - you would be better off looking at a specific antiderivative. Namely, assume $P$ is defined on $[a,b]\times[c,d]$ and let
$$Q(x,y):=\int_a^xP(t,y)\,dt,$$
so $\frac{\partial Q}{\partial x}(x,y)=P(x,y)$. Your question becomes about whether we have
$$\frac{\partial Q}{\partial y}(x,y)=\int_a^x\frac{\partial P}{\partial y}(t,y)\,dt.$$
This is, of course, a question about interchanging limits, and as in your proof, the bounded convergence theorem is the way to go. We have
$$\frac{Q(x,y+h)-Q(x,y)}{h}=\int_a^xR_h(t,y)\,dt$$
where $R_h(t,y)=\frac{P(t,y+h)-P(t,y)}h$. Clearly $R_h(t,y)\to\frac{\partial P}{\partial y}(t,y)$ as $h\to0$. Moreover, by the intermediate value theorem, $R_h(t,y)=\frac{\partial P}{\partial y}(t,c)$ for some $c$ between $y$ and $y+h$. Since $P$ is $C^1$, $\frac{\partial P}{\partial y}$ is bounded. This is enough to apply the bounded convergence theorem and conclude.
In terms of Riemann vs Lebesgue integral, I would say as a general rule you should always assume you're working with the Lebesgue integral (once you know it) except for very specific circumstances, such as discussing the convergence of an improper integral which is not absolutely convergent. It is simple to show that if $f$ is Riemann integrable (on a bounded interval) then it is also Lebesgue integrable, and we have many more tools available with Lebesgue integration, so we may as well use them.
Best Answer
You may interchange integration and differentiation precisely when Leibniz says you may. In your notation, for Riemann integrals: when $f$ and $\frac{\partial f(x,t)}{\partial x}$ are continuous in $x$ and $t$ (both) in an open neighborhood of $\{x\} \times [a,b]$.
There is a similar statement for Lebesgue integrals.