The full question reads:
Show that the system with $\dot{x} = x-y-(x^2 + \frac{3}{2} y^2)x$ and $\dot{y} = x+y-(x^2 + \frac{1}{2} y^2)y$ has a limit cycle.
I wanted to check the correctness of my method. My thought was to find the equilibrium points (If I am correct, the only one is $(0,0)$, then find the stability of that point. Then I attempted to show that if we go far enough out in the domain, that the flow of the direction field is going in the opposite direction of the direction field directly surrounding the equilibrium point. Then we can use Poincare-Bendixon and be done. Is this correct, or should I handle it differently? Thanks!
Best Answer
Another approach is to proceed using the Poincare-Bendixon theorem. Our aim then is to show that the system contains only one unstable equilibrium point, and that every trajectory of the system remains in a closed bounded subset.
From your original problem, let $$ f_1(x,y) = x - y - x^3 - \frac{3}{2}y^2{x} \nonumber \\ f_2(x,y) = x + y - x^2{y} - \frac{1}{2}y^3 \nonumber \\ $$ Then $$ \dot{x} = f_1(x,y) \nonumber \\ \dot{y} =f_2(x,y) \nonumber \\ $$
The equilibrium point is $x=0$ and $y =0$
The Jacobian is
\begin{align} J &= \begin{bmatrix} \frac{\partial{f_1}}{\partial x} & \frac{\partial{f_1}}{\partial y} \\ \frac{\partial{f_2}}{\partial y} & \frac{\partial{f_2}}{\partial y} \\ \end{bmatrix} \nonumber _{\substack{x=0 \\ y=0}} \\ &= \begin{bmatrix} 1 & -1 \\ 1 & 1 \\ \end{bmatrix} \end{align}
The eigenvalues are $1+i$ and $1-i$. Hence the system is unable (unstable node or focus), which is the first criterion of the Poincare-Bendixon theorem.
Secondly, let us choose a closed bounded subset given by $$ V(x,y)=x^2 + y^2 \le c \nonumber $$ where c is a positive constant. Therefore the chosed subset $V(x,y)$ is a circle of radius $\sqrt c$.
We only need to show that there exists a finte value of $c$ for which the vector field of $f(x,y)$ never leaves the set enclosed by $c$. This is identical to the statement: $$ \nabla {V(x,y)} \cdot f(x,y) \le 0 \nonumber $$
Now, $$ \nabla {V(x,y)} \cdot f(x,y) = \frac{\partial V}{\partial x}f_1(x,y) + \frac{\partial V}{\partial y}f_2(x,y) \nonumber $$
Therefore $$ \frac{\partial V}{\partial x}f_1(x,y) + \frac{\partial V}{\partial y}f_2(x,y) \nonumber $$ $$ = 2x\left[x-y-\left(x^2+\frac{3}{2}y^2\right)x\right] + 2y\left[x+y-\left(x^2+\frac{1}{2}y^2\right)y\right] \nonumber \\ = 2{\left(x^2+y^2\right)} -2x^2{\left(x^2+y^2+\frac{1}{2}y^2\right)}-2y^2{\left(x^2+y^2-\frac{1}{2}y^2\right)} \\ = 2{\left(x^2+y^2\right)} - 2\left(x^2+y^2\right)^2 + y^4-x^2y^2 $$
But $$ y^4-x^2y^2 \le \left(x^2+y^2\right)^2 $$
Therefore $$ 2{\left(x^2+y^2\right)} - 2\left(x^2+y^2\right)^2 + y^4-x^2y^2 \le 2{\left(x^2+y^2\right)} - 2\left(x^2+y^2\right)^2 +\left(x^2+y^2\right)^2 = 2{\left(x^2+y^2\right)} - \left(x^2+y^2\right)^2 $$
This implies that $$ \frac{\partial V}{\partial x}f_1(x,y) + \frac{\partial V}{\partial y}f_2(x,y) \le 2{\left(x^2+y^2\right)} - \left(x^2+y^2\right)^2 = 2c - c^2 = (2-c)c \nonumber $$
Finally, if we choose $c=2$, we are guaranteed that $$ \frac{\partial V}{\partial x}f_1(x,y) + \frac{\partial V}{\partial y}f_2(x,y) \le 0 \nonumber $$ which satisfies the second criterion of the Poincare-Bendixon theorem.
Therefore, it can be concluded that a limit cycle exists.