Poincare-Bendixson states that if the $\omega$-limit set of the trajectory doesn't contain critical point, then the $\omega$-limit set is a periodic orbit, I am just wondering if there is an example showing that $\omega$-limit set contains both critical point and periodic orbit
Poincare Bendixson theorem
ordinary differential equations
Related Solutions
I interpreted the question as follow:
Let $f : \mathbb{R}^2 \to \mathbb{R}^2$ be a $\mathcal{C}^1$ vector field. Suppose there exists a periodic orbit $\gamma$. According to Jordan-curve-theorem, $\mathbb{R}^2 \backslash \gamma$ admits only one bounded connected component $D_{\gamma}$. Show that $D_{\gamma}$ contains a fixed point.
In fact, you only need a weak version of Poincaré-Bendixson theorem:
Theorem: Let $f : \mathbb{R}^2 \to \mathbb{R}^2$ be a $\mathcal{C}^1$ vector field and $x \in \mathbb{R}^2$. If the $\omega$-limit $\omega(x)$ is nonempty, compact and does not contain any fixed point, then $\omega(x)$ is a periodic orbit.
Notice that it is also true for $\alpha$-limits, since by reversing time an $\alpha$-limit becomes an $\omega$-limit while the phase portrait is unchanged.
By contradiction, suppose $D_{\gamma}$ does not contain any fixed point. Then, by Poincaré-Bendixson theorem, for all $x \in D_{\gamma}$, $\omega(x)$ and $\alpha(x)$ are periodic orbits. In particular, there are infinitely many periodic orbits in $D_{\gamma}$.
For any periodic orbit $\tau$ in $D_{\gamma}$, let $K_{\tau}=\tau \cup D_{\tau}$ (where $D_{\tau}$ is defined like $D_{\gamma}$ but for $\tau$). We get a family of compacts $\{K_i : i \in I \}$ linearly ordered by inclusion, indexed by some unbounded set $I \subset \mathbb{R}_+$. Without loss of generality, we can suppose the family $\{K_i : i \in I\}$ nonincreasing (otherwise, take $\tilde{K}_i= \bigcap\limits_{j \leq i} K_j$).
Because any $x \in \mathbb{R}^2$ is between its $\omega$-limit and its $\alpha$-limit, $\bigcap\limits_{i \in I} K_i=\emptyset$.
Now, take $i_n \in I$ such that $i_n \underset{n\to + \infty}{\longrightarrow} + \infty$; then $\bigcap\limits_{n \geq 0} K_{i_n}=\emptyset$. So we find a nonincreasing sequence of compacts converging to the emptyset, a contradiction (since $\mathbb{R}^2$ is complete).
Best Answer
Trivially, a critical point is also a periodic orbit. However, I believe that for any periodic orbits containing more than one point, you reach a contradiction. Recall that a point $x$ is considered an element of a periodic orbit if there exists a positive time $t$ such that $\Phi_t(x) = x$.
Now suppose both $x_1$ and $x_2$ are elements of a periodic orbit and that $x_1$ is a critical point. Obviously if the flow ever reaches $x_1$, it remains there forever, contradicting the fact that $x_2$ is in the $\omega$-limit set. However, if the flow never reaches $x_1$, then we contradict the fact that $x_1$ is in the $\omega$-limit set. Therefore, we reach a contradiction if there is a critical point in the periodic orbit.