I interpreted the question as follow:
Let $f : \mathbb{R}^2 \to \mathbb{R}^2$ be a $\mathcal{C}^1$ vector field. Suppose there exists a periodic orbit $\gamma$. According to Jordan-curve-theorem, $\mathbb{R}^2 \backslash \gamma$ admits only one bounded connected component $D_{\gamma}$. Show that $D_{\gamma}$ contains a fixed point.
In fact, you only need a weak version of Poincaré-Bendixson theorem:
Theorem: Let $f : \mathbb{R}^2 \to \mathbb{R}^2$ be a $\mathcal{C}^1$ vector field and $x \in \mathbb{R}^2$. If the $\omega$-limit $\omega(x)$ is nonempty, compact and does not contain any fixed point, then $\omega(x)$ is a periodic orbit.
Notice that it is also true for $\alpha$-limits, since by reversing time an $\alpha$-limit becomes an $\omega$-limit while the phase portrait is unchanged.
By contradiction, suppose $D_{\gamma}$ does not contain any fixed point. Then, by Poincaré-Bendixson theorem, for all $x \in D_{\gamma}$, $\omega(x)$ and $\alpha(x)$ are periodic orbits. In particular, there are infinitely many periodic orbits in $D_{\gamma}$.
For any periodic orbit $\tau$ in $D_{\gamma}$, let $K_{\tau}=\tau \cup D_{\tau}$ (where $D_{\tau}$ is defined like $D_{\gamma}$ but for $\tau$). We get a family of compacts $\{K_i : i \in I \}$ linearly ordered by inclusion, indexed by some unbounded set $I \subset \mathbb{R}_+$. Without loss of generality, we can suppose the family $\{K_i : i \in I\}$ nonincreasing (otherwise, take $\tilde{K}_i= \bigcap\limits_{j \leq i} K_j$).
Because any $x \in \mathbb{R}^2$ is between its $\omega$-limit and its $\alpha$-limit, $\bigcap\limits_{i \in I} K_i=\emptyset$.
Now, take $i_n \in I$ such that $i_n \underset{n\to + \infty}{\longrightarrow} + \infty$; then $\bigcap\limits_{n \geq 0} K_{i_n}=\emptyset$. So we find a nonincreasing sequence of compacts converging to the emptyset, a contradiction (since $\mathbb{R}^2$ is complete).
These two versions of the Poincaré-Bendixson Theorem are not contradicting each other. Instead, I think you overlooked some of the hypotheses.
The first theorem tells you that a positively invariant, compact subset of the phase plane always contains at least one closed orbit, provided there are no fixed points in it (or provided it has just one unstable node or spiral point in its interior). While the proof of this fact relies on the topology of $\mathbb{R}^2$ a lot, the idea behind its formulation is easy to understand: if a trajectory is trapped in a compact subset of the phase plane which does not contain any stable fixed points, then it would have nothing to do but to approach a limit cycle.
The second theorem relies on different hypotheses: $R$ is still a positively invariant, compact subset of the phase plane, but it does not contain any periodic solutions now. So you should ask yourself: if a solution is forever trapped in $R$, and there are no limit cycles to be approached, what can that solution possibly do? Flowing toward a stable equilibrium point, of course. Indeed, this answers your question: it is possible to have a region $R$ that is a positively invariant, compact subset of the phase plane, with no periodic solutions in it, that still contains a finite number of nodes or spiral points: you'd just need some of those fixed points to be stable.
Also, pay attention to the fact your book says nothing about a positively invariant, compact subset of the phase plane that is both free from equilibrium points and limit cycles. Such a region cannot exist, and this is the very essence of the Poincaré-Bendixson Theorem: once you manage to trap a solution inside a region like $R$, then it can only either approach a (stable) fixed point, or spiral toward a limit cycle.
There's actually a third possibility, that of approaching a more complicated object called a cycle graph (which consists of a finite number of fixed points together with the orbits connecting them). Yet the spirit of the theorem does not change: a bounded solution may not wander in a chaotic fashion, because nothing different from approaching a stable fixed point, a limit cycle or a cycle graph is allowed. As a matter of fact, the Poincaré-Bendixson Theorem is what tells you that chaos can never occur in a two-dimensional dynamical system.
Best Answer
The hint says that all equilibria are on the axes. After some computations you will find that these are $$ (0,-1), \quad (0,0),\quad (0,1),\quad (-2^{1/4}, 0),\quad (2^{1/4}, 0). $$ As you wrote, if the flow has a periodic orbit, then the region that it bounds contains an equilibrium. So necessarily the orbit must cross at least one of the axes since this is the only way that one of those $5$ points is "inside" the region bounded by the periodic orbit.
But both axes are invariant and so no orbit can cross them. Indeed, for the horizontal axis, taking $y=0$ the second equation is automatically verified and so the flow is obtained solving $\dot x=2x-x^5$ (which is possible because it does not depend on $y$). A similar observation applies to the vertical axis.
In particular, there are no periodic orbits.