This is a community-wiki answer trying to remove this question from the unanswered queue.
Question 1:
$curl(E)=[2x,0,−2x].$
This is not correct. If $\mathbf{E} = (3yz,zx,2xy)$, then
$$
\nabla \times \mathbf{E} = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\
\partial_x &\partial_y & \partial_z \\ 3yz & zx & 2xy \end{vmatrix}
= (2x-x, 3y-2y, z-3z) = (x,y,-2z).
$$
Hence restricted on this surface $z=4-x^2-y^2$ we have:
$$
\nabla \times \mathbf{E}\big\vert_{S} = (x,y, 2x^2+2y^2-8).
$$
$ds=[-2x,-2y,1]$.
There is a minor sign error in this, notice
$$
d\mathbf{S} = \left(-\frac{\partial z}{\partial x}, -\frac{\partial z}{\partial y}, 1\right) = (2x,2y,1).
$$
Double integral $\iint(-4x^2-2x ) \,dxdy$ into polar coordinates $\int_0^{2\pi}\int_0^2 (-4(r\cos\theta)^2-2(r\cos\theta))rdrd\theta$.
$x,y\geq 0$, and $x^2+y^2\leq 4$, hence the surface is defined in the first octant , and the range for $x,y$ should be the first quadrant in the $xy$-plane. The range of the polar angle should be from $0$ to only $\pi/2$. To combine all the results above:
$$
\oint_C \mathbf{E}\cdot \mathbf{t} \,ds =
\iint_S\nabla \times \mathbf{E}\cdot d\mathbf{S}
\\
= \iint_{\{x,y\geq 0 : \; x^2+y^2\leq 4\}} (2x^2+2y^2 + 2x^2+2y^2-8 ) \,dxdy
\\
=\int^{\pi/2}_0 \int^2_0 (4r^2-8)r\,drd\theta = 0.$$
Question 2:
So my triple integral is now $\int_0^3\int_0^{2\pi}\int_0^\pi (3p^2)p^2sin(\phi)d\phi d\theta dp$ = $\frac{2916\pi}{5}$.
The polar integral set up is perfect, just I suggest use $r$ as the radial variable:
$$
\int_0^3\int_0^{2\pi}\int_0^\pi (3r^2)r^2\sin \phi \,d\phi d\theta dr.
$$
$\underline r = (x, y, z)$ and not $(x,y,a^2-x^2-y^2)$.
$z = a^2 - x^2 - y^2$ only on the paraboloid surface.
$\nabla \cdot \underline r = 3$
In polar coordinates, $0 \leq z \leq a^2 - \rho^2, 0 \leq \rho \leq a$
So the volume integral is,
$ \displaystyle \int_0^{2\pi} \int_0^{a^2} \int_0^{\sqrt{a^2 - z}} 3~\rho~ d\rho~ dz~d\theta = \frac{3 \pi a^4}{2}$
Best Answer
You need to use cylindrical coordinates, as the surface has cylindrical symmetry. The integral you want is
$$3 \int_0^{a^2} dz \: \int_0^{a^2-z} dr \,r \: \int_0^{2 \pi} d\theta $$
Note that the volume element in cylindrical coordinates is $r\,dr\,d\theta\,dz$, and the bounds are explicitly set by the equation of the paraboloid. The $3$ comes from your divergence. You should be able to do out this integral.
My answer for this integral is $3 \pi a^4/2 $.