Let $\Omega = \{ (x,y,z) \;|\; (x,y) \in D \mbox{ and } 0 \leq z \leq 1-x-y \}$. Then turn the triple integral into a double $+$ a single integral...
$$(p-1)\iiint\limits_{\Omega} x^{m-1}y^{n-1}z^{p-2} \,dV = \iint\limits_D \left(\int_0^{1-x-y} (p-1)x^{m-1}y^{n-1}z^{p-2}\,dz\right)\,dA$$
$$= \iint_D \left. x^{m-1}y^{n-1}z^{p-1} \right|_0^{1-x-y}\,dA =
\iint_D x^{m-1}y^{n-1}(1-x-y)^{p-1}\,dA$$
$ \displaystyle {\int_{-1}^1} {\int_{-\sqrt {1-x^2}}^{\sqrt {1-x^2}}} {\int_{- \sqrt{x^2 + y^2}}^1} \ dz \ dy \ dx$
Note the region is bound by the following surfaces:
Cone $z = - \sqrt{x^2+y^2}$, cylinder $x^2+y^2 = 1$ and plane $z = 1$.
$x = \rho \cos\theta \sin\phi, y = \rho \sin\theta \sin\phi, z = \rho \cos\phi$
If we are integrating wrt $\rho$ first, we have to split the integral into two -
i) when $\rho$ is bound by the cylinder
$x^2 + y^2 \leq 1 \implies \rho \leq \csc\phi$
Using $z = - \sqrt{x^2+y^2}$, upper bound of $\phi = \cfrac{3\pi}{4}$
Lower bound of $\phi$ is at the intersection of cylinder and plane $z = 1$.
$z = 1 \implies \rho = \sec \phi$
So at intersection, $\sec\phi = \csc\phi \implies \phi = \cfrac{\pi}{4}$
ii) when $\rho$ is bound by the plane $z = 1$
$\rho \leq \sec\phi, 0 \leq \phi \leq \cfrac{\pi}{4}$
So integral is,
$\displaystyle \int_0^{2\pi} \int_{\pi/4}^{3 \pi/4} \int_0^{\csc\phi} \rho^2 \sin \phi \ d\rho \ d\phi \ d\theta \ $ +
$\displaystyle \int_0^{2\pi} \int_{0}^{\pi/4} \int_0^{\sec\phi} \rho^2 \sin \phi \ d\rho \ d\phi \ d\theta$
Best Answer
Ok so if you want regular spherical coordinates: $x^2+y^2+z^2=\rho^2,$ and $2y=2\rho\sin(\theta)\sin(\phi).$
So the only information about limits you have is $\rho^2=2\rho\sin(\theta)\sin(\phi).$ One solution is $\rho=0.$ The other is $\rho=2\sin(\theta)\sin(\phi),$ as you said. Now it's a tiny bit trickier. You get the next bounds by setting the $\rho$ bounds equal: $0=2\sin(\theta)\sin(\phi).$ The solutions are $\theta=n\pi$ or $\phi=0$ or $\phi=\pi.$ So $0\le \phi\le \pi$ is correct, and you have $0\le \theta\le\pi$ as well.
The integral becomes $$\int_0^\pi \int_0^\pi \int_0^{2\sin(\theta)\sin(\phi)} \rho^3 \sin(\theta)\sin^2(\phi)\,d\rho\,d\theta\,d\phi=4\int_0^\pi \int_0^\pi \sin^5(\theta)\sin^6(\phi)\,d\theta\,d\phi.$$ Now I'll just use the facts $\int_0^\pi \sin^5(x)\,dx=16/15$ and $\int_0^\pi \sin^6(x)\,dx=5\pi/16$ to get the final answer.
You can check this with the easier integral using the additional substitution of $u=y-1:$
$$\int_0^{2\pi}\int_0^\pi\int_0^1 (\rho\sin(\phi)\sin(\theta)+1)\rho^2\sin(\phi)\,d\rho\,d\phi\,d\theta$$ $$=\frac{4\pi}{3}+\int_0^{2\pi}\int_0^\pi\int_0^1 \rho^3\sin^2(\phi)\sin(\theta)\,d\rho\,d\phi\,d\theta=\frac{4\pi}{3}$$ since the integral of $\sin(\theta)$ is zero on that interval. It's the same answer.