One way we could go is to subtract $x$ times the second equation from the first, getting us: $$xy^2 - 2y + 3x^2-x(x^2y + 2x + y^2) = 0\\xy^2 - 2y + 3x^2-x^3y - 2x^2 -xy^2 = 0\\x^2-x^3y-2y=0\\x^2=x^3y+2y\\x^2=(x^3+2)y$$ Note that if we had $x^3+2=0,$ then the last equation above would become $x^2=0,$ so $x=0,$ contradicting our assumption that $x\ne 0.$ Thus, we don't have to worry about $x^3+2$ being zero, and so $$y=\frac{x^2}{x^3+2}.\tag{$\star$}$$ Substituting for $y$ in the second equation gets us $$x^2\cdot\frac{x^2}{x^3+2}+2x+\left(\frac{x^2}{x^3+2}\right)^2=0,$$ or $$x\left(\frac{x^3}{x^3+2}+\frac{x^3}{(x^3+2)^2}+2\right)=0,$$ which (since $x\ne 0$) is equivalent to $$\frac{x^3}{x^3+2}+\frac{x^3}{(x^3+2)^2}+2=0\\x^3(x^3+2)+x^3+2(x^3+2)^2=0\\x^6+3x^3+2(x^6+4x^3+4)=0\\3x^6+11x^3+8=0\\3\left(x^3\right)^2+11x^3+8=0\\x^3=\frac{-11\pm\sqrt{121-96}}6\\x^3=\frac{-11\pm \sqrt{25}}6\\x^3=\frac{-11\pm 5}6\\x^3=-1\textrm{ or }x^3=-\frac83$$
If we're looking for real solutions, then we have $x=-1$ or $x=-\frac{2}{\sqrt[3]3},$ but if we're interested in non-real solutions, too, then since $-\frac12\pm\frac{\sqrt3}2$ are cube roots of $-1,$ too, we also have $x=\frac12\pm\frac{\sqrt3}2$ and $x=-\frac{2}{\sqrt[3]3}\left(-\frac12\pm\frac{\sqrt3}2\right).$ (For brevity, I will denote $\omega=-\frac12+\frac{\sqrt3}2$ and $\overline\omega=-\frac12-\frac{\sqrt3}2.$)
When $x\in\left\{-1,\omega,\overline\omega\right\},$ we have $x^3=-1,$ so $x^3+2=1,$ and so $(\star)$ becomes $$y=x^2.$$ Thus, we obtain the solutions $(-1,1),$ $\left(\omega,\omega^2\right)=\left(\omega,-\overline\omega\right),$ and $\left(\overline\omega,\overline\omega^2\right)=\left(\overline\omega,-\omega\right).$
When $x\in\left\{-\frac{2}{\sqrt[3]3},-\frac{2\omega}{\sqrt[3]3},-\frac{2\overline\omega}{\sqrt[3]3}\right\},$ we have $x^3=-\frac83,$ so $x^3+2=-\frac23,$ and so $(\star)$ becomes $$y=-\frac32x^2.$$ Thus, we obtain the solutions $\left(-\frac{2}{\sqrt[3]3},-2\sqrt[3]3\right),$ $\left(-\frac{2\omega}{\sqrt[3]3},2\overline\omega\sqrt[3]3\right),$ and $\left(-\frac{2\overline\omega}{\sqrt[3]3},2\omega\sqrt[3]3\right).$
In summary, our solutions are: $$(0,0),(-1,1),\left(-\frac{2}{\sqrt[3]3},-2\sqrt[3]3\right),\left(\omega,-\overline\omega\right),\left(\overline\omega,-\omega\right),\left(-\frac{2\omega}{\sqrt[3]3},2\overline\omega\sqrt[3]3\right),\left(-\frac{2\overline\omega}{\sqrt[3]3},2\omega\sqrt[3]3\right),$$ with the first three being the real solutions.
Best Answer
Let $\sigma=x^2+1$, note that $\sigma y\ne 0$. The first equation is equivalent to: $$\sigma+y(x+y)-3y=0\tag1$$ and the second: $$x+y-1=\frac{y}\sigma\tag2$$
$$(1)\iff x+y=3-\frac{\sigma}{y}.$$ Replacing this last one into $(2)$ yields: $$2=\frac{y}\sigma+\frac{\sigma}y\iff (y-\sigma)^2=0.$$ Therefore $y=\sigma=x^2+1=2-x$. I let you take it from here.