Can someone sketch some ideas of how to use the Poincaré-Bendixson Theorem to prove that there must be a fixed point contained inside a periodic orbit?
[Math] Poincare-Bendixson Theorem
dynamical systemsordinary differential equations
Related Solutions
These two versions of the Poincaré-Bendixson Theorem are not contradicting each other. Instead, I think you overlooked some of the hypotheses.
The first theorem tells you that a positively invariant, compact subset of the phase plane always contains at least one closed orbit, provided there are no fixed points in it (or provided it has just one unstable node or spiral point in its interior). While the proof of this fact relies on the topology of $\mathbb{R}^2$ a lot, the idea behind its formulation is easy to understand: if a trajectory is trapped in a compact subset of the phase plane which does not contain any stable fixed points, then it would have nothing to do but to approach a limit cycle.
The second theorem relies on different hypotheses: $R$ is still a positively invariant, compact subset of the phase plane, but it does not contain any periodic solutions now. So you should ask yourself: if a solution is forever trapped in $R$, and there are no limit cycles to be approached, what can that solution possibly do? Flowing toward a stable equilibrium point, of course. Indeed, this answers your question: it is possible to have a region $R$ that is a positively invariant, compact subset of the phase plane, with no periodic solutions in it, that still contains a finite number of nodes or spiral points: you'd just need some of those fixed points to be stable.
Also, pay attention to the fact your book says nothing about a positively invariant, compact subset of the phase plane that is both free from equilibrium points and limit cycles. Such a region cannot exist, and this is the very essence of the Poincaré-Bendixson Theorem: once you manage to trap a solution inside a region like $R$, then it can only either approach a (stable) fixed point, or spiral toward a limit cycle.
There's actually a third possibility, that of approaching a more complicated object called a cycle graph (which consists of a finite number of fixed points together with the orbits connecting them). Yet the spirit of the theorem does not change: a bounded solution may not wander in a chaotic fashion, because nothing different from approaching a stable fixed point, a limit cycle or a cycle graph is allowed. As a matter of fact, the Poincaré-Bendixson Theorem is what tells you that chaos can never occur in a two-dimensional dynamical system.
Trivially, a critical point is also a periodic orbit. However, I believe that for any periodic orbits containing more than one point, you reach a contradiction. Recall that a point $x$ is considered an element of a periodic orbit if there exists a positive time $t$ such that $\Phi_t(x) = x$.
Now suppose both $x_1$ and $x_2$ are elements of a periodic orbit and that $x_1$ is a critical point. Obviously if the flow ever reaches $x_1$, it remains there forever, contradicting the fact that $x_2$ is in the $\omega$-limit set. However, if the flow never reaches $x_1$, then we contradict the fact that $x_1$ is in the $\omega$-limit set. Therefore, we reach a contradiction if there is a critical point in the periodic orbit.
Best Answer
I interpreted the question as follow:
Let $f : \mathbb{R}^2 \to \mathbb{R}^2$ be a $\mathcal{C}^1$ vector field. Suppose there exists a periodic orbit $\gamma$. According to Jordan-curve-theorem, $\mathbb{R}^2 \backslash \gamma$ admits only one bounded connected component $D_{\gamma}$. Show that $D_{\gamma}$ contains a fixed point.
In fact, you only need a weak version of Poincaré-Bendixson theorem:
Notice that it is also true for $\alpha$-limits, since by reversing time an $\alpha$-limit becomes an $\omega$-limit while the phase portrait is unchanged.
By contradiction, suppose $D_{\gamma}$ does not contain any fixed point. Then, by Poincaré-Bendixson theorem, for all $x \in D_{\gamma}$, $\omega(x)$ and $\alpha(x)$ are periodic orbits. In particular, there are infinitely many periodic orbits in $D_{\gamma}$.
For any periodic orbit $\tau$ in $D_{\gamma}$, let $K_{\tau}=\tau \cup D_{\tau}$ (where $D_{\tau}$ is defined like $D_{\gamma}$ but for $\tau$). We get a family of compacts $\{K_i : i \in I \}$ linearly ordered by inclusion, indexed by some unbounded set $I \subset \mathbb{R}_+$. Without loss of generality, we can suppose the family $\{K_i : i \in I\}$ nonincreasing (otherwise, take $\tilde{K}_i= \bigcap\limits_{j \leq i} K_j$).
Because any $x \in \mathbb{R}^2$ is between its $\omega$-limit and its $\alpha$-limit, $\bigcap\limits_{i \in I} K_i=\emptyset$.
Now, take $i_n \in I$ such that $i_n \underset{n\to + \infty}{\longrightarrow} + \infty$; then $\bigcap\limits_{n \geq 0} K_{i_n}=\emptyset$. So we find a nonincreasing sequence of compacts converging to the emptyset, a contradiction (since $\mathbb{R}^2$ is complete).