The surface $S$ is the part of the plane $z=4+x+y$ that lies inside the cylinder $x^2+y^2=4$. We want to describe $S$ as a parametric surface $\vec r=\vec r(u,v)$. This is most easily accomplished in cylindrical coordinates, Where the equation of the plane takes the form $z=4+\rho\cos{\phi}+\rho\sin{\phi}=4+\rho(\cos{\phi}+\sin{\phi})$, and the equation of the cylinder takes the form $\rho=2$.
We have the following parametric representation of $S$:
$$\vec{r}(\rho,\phi)=\langle\rho\cos{\phi},\rho\sin{\phi},4+\rho(\cos{\phi}+\sin{\phi})\rangle,~~~0\leq\rho\leq2,0\leq\phi\leq2\pi.$$
Computing the partial derivatives and their cross product, we get
$$\vec{r}_\rho=\langle\cos{\phi},\sin{\phi},\cos{\phi}+\sin{\phi}\rangle$$
$$\vec{r}_\phi=\langle-\rho\sin{\phi},\rho\cos{\phi},\rho(\cos{\phi}-\sin{\phi})\rangle$$
$$\vec{r}_\rho\times\vec{r}_\phi=\langle-\rho,-\rho,\rho\rangle.$$
Thus, $\|\vec{r}_\rho\times\vec{r}_\phi\|=\sqrt{3}\rho$.
Rewriting the integrand in cylindrical coordinates, we have
$$f(x,y,z)=x^2z+y^2z=(x^2+y^2)z=\rho^2z.$$
Recall that on the surface $S$, we have $z=4+\rho(\cos{\phi}+\sin{\phi})$, so then
$$f(\vec{r}(\rho,\phi))=\rho^2(4+\rho(\cos{\phi}+\sin{\phi})).$$
Putting it all together, the surface integral evaluates to
$$I=\iint_{S}f(\vec{r})\,dS=\iint_{D}f(\vec{r}(\rho,\phi))\|\vec{r}_\rho\times\vec{r}_\phi\|\,dA\\
=\sqrt{3}\int_{0}^{2\pi}\int_{0}^{2}\rho^3(4+\rho(\cos{\phi}+\sin{\phi}))\,d\rho\,d\phi$$
We can make the evaluation of the above integral even easier by switching the order of integration and noting that $\cos\phi+\sin\phi$ is periodic in $\phi$ with period $2\pi$ and that the integral over one period is automatically zero. Hence, the value of the surface integral is:
$$I=\sqrt{3}\int_{0}^{2\pi}\int_{0}^{2}\rho^3(4+\rho(\cos{\phi}+\sin{\phi}))\,d\rho\,d\phi=\sqrt{3}\int_{0}^{2}\int_{0}^{2\pi}(4\rho^3+\rho^4(\cos{\phi}+\sin{\phi}))\,d\phi\,d\rho\\
=\sqrt{3}\int_{0}^{2}\int_{0}^{2\pi}4\rho^3\,d\phi\,d\rho\\
=2\sqrt{3}\pi\int_{0}^{2}4\rho^3\,d\rho\\
=2\sqrt{3}\pi(2^4)\\
=2^5\sqrt{3}\pi.$$
Best Answer
To calculate a surface integral over a vector field you need parametrization of that oriented surface. Without too much of rigour:
$$\begin{align}\iint_S {\mathbf v}\cdot\mathrm d{\mathbf {S}} &= \iint_S \left({\mathbf v}\cdot {\mathbf n}\right)\,\mathrm dS\\&{}= \iint_T \left({\mathbf v}(\mathbf{x}(s, t)) \cdot {\left({\partial \mathbf{x} \over \partial s}\times {\partial \mathbf{x} \over \partial t}\right) \over \left\|\left({\partial \mathbf{x} \over \partial s}\times {\partial \mathbf{x} \over \partial t}\right)\right\|}\right) \left\|\left({\partial \mathbf{x} \over \partial s}\times {\partial \mathbf{x} \over \partial t}\right)\right\| \mathrm ds\, \mathrm dt\\&{}=\iint_T {\mathbf v}(\mathbf{x}(s, t))\cdot \left({\partial \mathbf{x} \over \partial s}\times {\partial \mathbf{x} \over \partial t}\right) \mathrm ds\, \mathrm dt.\end{align}$$
$d{\mathbf {S}}$ means that you are integrating vector field over an oriented surface. When you "extract" a unit normal vector from it you are left with a scalar function and $dS$ which correspond to a "normal" (scalar) surface integral. In this notation $$dS= \left\|\left({\partial \mathbf{x} \over \partial s}\times {\partial \mathbf{x} \over \partial t}\right)\right\| \mathrm ds\, \mathrm dt$$
where $\mathbf{x}(s,t): T\rightarrow S$ is a parametrization of your surface.
Also please note how $\left\|\left({\partial \mathbf{x} \over \partial s}\times {\partial \mathbf{x} \over \partial t}\right)\right\| $ cancel each other and simplify calculations.