First, lets see where both surfaces intersect. In polar coordinates, we have
$$
z=3-r = 1+ r^2\quad \Rightarrow r=1.
$$
So the intersection is the circle $r=1$ at height $z=2$.
The area of your surface is the area $A_1$ of $z=3-\sqrt{x^2+y^2}$ above this circle (let $S_1$ be this surface), plus the area $A_2$ of $z=1+x^2+y^2$ beneath this circle (let $S_2$ be this surface).
You can parametrize $S_1$ with the vectorial function $f$:
$$
x=x, \quad y=y, \quad z=3-\sqrt{x^2+y^2},
$$
and $S_2$ with $g$:
$$
x=x, \quad y=y, \quad z=3+x^2+y^2,
$$
with $ (x,y)\in D=\{(r,\theta)\;|\;0\le r\le 1, 0\le \theta \le 2\pi\}$. Therefore, the total area equals
$$
A_1+A_2=\iint_D ||f_x\times f_y|| dA + \iint_D ||g_x\times g_y|| dA\\
=\sqrt{2}\int_0^{2\pi}\int_{0}^1rdrd\theta+\int_0^{2\pi}\int_{0}^1r\sqrt{4r^2+1}drd\theta\\
=\sqrt{2}\pi+\frac{5\sqrt{5}-1}{6}\pi
$$
Option 1:
The projection of the intersection of $z=x^2+y^2$ and $z=y$ in the $xy$ plane is $x^2+y^2=y$, i.e., the circle centered at $(0,1/2)$ with radius $1/2$, or in polar coordinates, $r=\sin \theta$. So you can parametrize the surface $S$ as follows:
$$
\begin{cases}
x=x \\
y=y \quad \quad \quad \quad \mbox{with }(x,y)\in D=\{(r,\theta)\;|\; 0 \le \theta \le \pi, \; 0 \le r \le \sin \theta \}\\
z=x^2+y^2
\end{cases}
$$
It follows that
$$
\Phi = \iint_S \nabla \times \vec{v} \; dS = \iint_D \nabla \times \vec{v}(x,y)\cdot \pmatrix{1\\0\\2x}\times \pmatrix{0\\1\\2y}\; dA = \iint_D \pmatrix{1\\2\\1} \cdot \pmatrix{-2x\\-2y\\1}\; dA \\= \iint_D 1-2x-4y \;dA
$$
This matches your answer, except for the $4$ coefficient for $y$. If I am not mistaken, $ \nabla \times \vec{v}=(1,2,1)$. Now what you need to do is integrate over $D$ (or $A$ with your notation), as defined above.
Switching to polar coordinates yields
$$
\Phi = \int_0^{\pi}\int_0^{\sin \theta}(1-2r \cos \theta-4r\sin\theta) r dr d\theta = -\frac{\pi}{4}
$$
Option 2:
Alternatively, if you are familiar with the divergence theorem:
$$
\Phi = \iint_S \nabla \times \vec{v} \; dS = \underbrace{\iiint_V \nabla\cdot \nabla \times \vec{v} \; dV}_{=0} - \iint_{S_2} \nabla \times \vec{v} \; dS
$$
where $S_2$ is the surface that closes $S$, i.e., the part of the plane $z=y$ on top of $D$, which we can parametrize with
$$
\begin{cases}
x=x \\
y=y \quad \quad \quad \quad \mbox{with }(x,y)\in D=\{(r,\theta)\;|\; 0 \le \theta \le \pi, \; 0 \le r \le \sin \theta \}\\
z=y
\end{cases}
$$
It follows that
$$
\Phi = - \iint_{S_2} \nabla \times \vec{v} \; dS = - \iint_D \nabla \times \vec{v}(x,y)\cdot \pmatrix{1\\0\\0}\times \pmatrix{0\\1\\1}\; dA = - \iint_D \pmatrix{1\\2\\1} \cdot \pmatrix{0\\-1\\1}\; dA = \iint_D dA = \frac{\pi}{4}
$$
When we use this theorem, we conventionally parametrize the surface "outwards", i.e., in the opposition direction as in Option $1$, hence the sign difference.
Option 3:
Last but not least, you can use Stoke's theorem:
$$
\Phi = \iint_S \nabla \times \vec{v} \; dS = \oint_C \vec{v} \cdot d\vec{r}
$$
where $C$ is the curve at the intersection between the paraboloid and the plane. Using the parametrization proposed by @levap in the question that you asked earlier :
$$
\begin{cases}
x=\frac{\cos t}{2} \\
y=\frac{1+\sin t}{2} \quad \quad \quad \quad \mbox{with } t\in [0,2\pi] \\
z=\frac{1+\sin t}{2}
\end{cases}
$$
It follows that
$$
\Phi = \oint_C \vec{v} \cdot d\vec{r} = \int_0^{2\pi} \frac{1}{4} \pmatrix{2(1+\sin t)\\ 1+ \cos t \\ 1 + \sin t}\cdot \pmatrix{-\sin t\\ \cos t \\ \cos t}\; dt = -\frac{\pi}{4}
$$
Isn't Calculus fun ? :)
Best Answer
I assume the surface in question lies above the plane $z = 0;$ however, if not, you can easily adapt this solution to the case that the surface contains both the upper and lower nappe of the cone.
Graphing the two surfaces shows that the surface for which we would like to compute the surface area resembles an ice cream cone: it consists of the cap $\mathcal C$ of the paraboloid $z = \frac 1 2 - x^2 - y^2$ and part of the upper nappe $\mathcal N$ of the cone $z^2 = x^2 + y^2.$ Ultimately, we will seek to compute $$\iint_{\partial V} 2(x^2 + y^2) \, dS = \iint_\mathcal C 2(x^2 + y^2) \, dS + \iint_\mathcal N 2(x^2 + y^2) \, dS.$$
Observe that $\mathcal C$ and $\mathcal N$ intersect if and only if $z^2 = x^2 + y^2$ and $z = \frac 1 2 - x^2 - y^2$ if and only if $z = \frac 1 2 - z^2$ if and only if $z^2 + z - \frac 1 2 = 0$ if and only if $z = \frac{\sqrt 3 - 1}{2}.$ Consequently, we have that $$\mathcal C = \biggl \{(x, y, z) \,|\, z = \frac 1 2 - x^2 - y^2 \text{ and } \frac{\sqrt 3 - 1}{2} \leq z \leq \frac 1 2 \biggr \} \text{ and}$$ $$\mathcal N = \biggl \{(x, y, z) \,|\, z^2 = x^2 + y^2 \text{ and } 0 \leq z \leq \frac{\sqrt 3 - 1}{2} \biggr \}. \phantom{\text{ and butt }}$$ Geometrically, the cap is a "deformation" of disk in the $xy$-plane, so we may parametrize $\mathcal C$ by polar coordinates $F(r, \theta) = \bigl \langle r \cos \theta, r \sin \theta, \frac 1 2 - r^2 \bigr \rangle$ for $0 \leq r \leq \sqrt{1 - \frac{\sqrt 3}{2}}$ and $0 \leq \theta \leq 2 \pi.$
Likewise, the upper nappe $\mathcal N$ of the cone can be parametrized most easily by polar coordinates $G(r, \theta) = \langle r \cos \theta, r \sin \theta, r \rangle$ for $0 \leq r \leq \frac{\sqrt 3 - 1}{2}$ and $0 \leq \theta \leq 2 \pi.$
Can you finish the solution from here? Use the definition of the surface integrals $$\iint_\mathcal C 2(x^2 + y^2) \, dS = \int_0^{2 \pi} \int_0^{\sqrt{1 - \sqrt 3 /2}} 2r^2 ||F_r(r, \theta) \times F_\theta(r, \theta)|| \cdot r \, dr \, d \theta \text{ and}$$ $$\iint_\mathcal N 2(x^2 + y^2) \, dS = \int_0^{2 \pi} \int_0^{(\sqrt 3 - 1)/2} 2r^2 ||G_r(r, \theta) \times G_\theta(r, \theta)|| \cdot r \, dr \, d \theta. \phantom{\text{ and }}$$