Goldstein's "closed" means the orbiting body will eventually return to some point with the same velocity it had there previously; i.e. that the path will repeat itself exactly. Note that this can occur even in the case of precession: if the ratio between radial period and angular period is rational, the orbit will be closed. There is no precession only if this ratio is unity.
What Goldstein is saying is that in this point in the text, all he has shown is that the objects' separation $r$ will satisfy $r_1 < r < r_2$ for all time, and that this does not necessarily imply that the orbit will ever repeat itself. For example, we have not ruled out the possibility that $r$ could be periodic with period $\pi$, while $\theta$ could repeat every 3 units of time.
Other authors use "closed" or "elliptical" to mean Goldstein's "bounded," and "open" or "hyperbolic" to mean "unbounded." Part of the degeneracy in terms probably comes from the fact that for two ideal point masses in Newtonian mechanics, bounded orbits will be closed in Goldstein's sense. See the application of Bertrand's Theorem to the inverse square law.
I was so surprised by the types of graphs I saw for this that I felt compelled to add an answer. As mentioned in the Wikipedia article I linked to, there are two radii that are of interest in addition to the Schwarzschild radius $r_s$. Those radii are
- the "Innermost Stable Circular Orbit" (ISCO) $ r_{outer} \approx2a^{2}/r_{s}$ and
- the "Innermost Bound Circular Orbit" (IBCO) $r_{inner} \approx 3/2 r_{s}$.
It is clear that the inner radius is clearly defined for the BH, but the outer radius depends on the specific orbital parameters. By definition the Kerr-Spin-parameter $a=J/(mc)$, where $J$ is the angular momentum of the central black hole and $m$ is its mass.* This, along with most info here, assumes the particle orbiting the BH has a small mass by comparison.
Here is one paper's graphs, covered by New Scientist. These patterns reflect the general pattern of zoom-whirl behavior, which might also be called Homoclinic orbits. As far as I can tell, these happen when an orbit traverses the ISCO but does not traverse the IBCO. I take it that (I am assuming that) if an orbit does not traverse the ISCO it has a shape like what I posted in the question, which I think are formally called Keplerian orbits.
Can some of these orbit in the small radius (or "zoom") more than once before making back out to the whirl? As it turns out, yes. An orbit can zoom many times before whirling, up to infinity, as per this paper.
For this orbit of infinite zooms, it would get infinitely close to the IBCO. In fact, that shines a lot of light on the nature of the IBCO as an unstable but balanced orbit point. No orbit can exist closer than the IBCO because it will just spiral inward, like a toilet bowl. In fact, there are two ways to dismiss non-orbits, all initial conditions either orbit, fall into the event horizon eventually, or they escape.
Kerr black holes, two-body problem, and other complications
Obviously we should expect more orbit possibilities if the black hole is spinning. Everything I've cited so far seems to be specific to orbits of small masses (relative to the BH) and non-rotating BHs. A lot of the combinations are apparently still being studied.
Under some conditions, a transition to chaos happens (reference: presentation from the periodic table of BH orbits paper researchers). Exactly what those conditions are is something I'm still unclear on, as a result of me not fully understanding this material.
In the case of a rotating black hole, it would seem that you can even orbit one way, and then stop and start orbiting the other way. Reference: http://www.lsw.uni-heidelberg.de/users/mcamenzi/Kerr_Black_Holes.pdf
I have a hard time imagining how the above picture is consistent with the hyperbolic orbits I see in Wikipedia, as it does not show this direction change at all. Maybe someone else can shed light on the difference and how they can both be true, if that is the case.
Also, if the particle isn't in the plane of the equator of the Kerr black hole, it will move up and down in the direction of the axis of rotation, giving it fully 3D dynamics. Here are some examples.
You can imagine, this is just scratching the surface. Combine that graph with the crazy zoom-whirl graphs, maybe even make it a 2-body problem, and it would make for some pretty fun paths.
In some conditions, the orbit becomes a Homoclinic tangle, which I won't even post a picture of because it just looks like a big knot.
* clarification from further questions in Expression for distance of closest approach in Schwarzschild Geodesics
Best Answer
It is not enough to have $r(t)=r(t+\Delta t)$: you must simultaneously have $\phi(t)=\phi(t+\Delta t)$ where $\phi$ is the angular degree of freedom. To put it differently, your need the radial and angular periods $T_r$ and $T_\phi$ to be commensurate, i.e. their ratio must be a rational number $n/m$.
This is because you need the orbit, which is described by the coordinates $(r,\phi)$, to exactly retrace itself. If $T_r/T_\phi$ is irrational, then the orbit will never close exactly on itself, and will eventually fill every point of $(r,\phi)$ space between $r_{min}$ and $r_{max}$.
Not every type of potential can produce closed orbits. This is discussed, for instance, in Marion and Thornton's Classical Dynamics and in multiple textbooks at the same level.
Edit: Simply knowing the energy is not enough.
For instance, the potential $$ V(r)=-\frac{k}{r}-\frac{\lambda}{2r^2} $$ leads to the effective potential $V_{eff}=\frac{\ell^2-\lambda m}{2 m r^2}-\frac{k}{r}$.
If $\lambda<\ell^2/\mu$ and $-mk^2/(2(\ell^2-m\lambda))\le E\le 0$ so we have bound states, the motion is described by a precessing ellipse, with $$ \phi=\phi_0-\frac{1}{\beta}\arccos\left(\frac{k-u(\beta^2\ell^2/{m})}{\sqrt{k^2+(2\beta^2\ell^2E/m)}}\right) $$ with $u=1/r$ and $\beta^2=1-m\lambda/\ell^2>0$ or, in terms of $r$: $$ r(\phi)=\frac{r_0}{1-e\cos\beta(\phi-\phi_0)} $$ for some $r_0$ and $e$. $r_0$ and $e$ can be expressed in terms of the other quantities in the problem. If $\beta$ is rational then the ratio of angular to radial period is given by $T_\phi/T_r=\beta$ and is also rational.
For instance, using $r_0=1$, the first two figures below show closed orbits, with $(e,\beta)=(\frac{4}{5},\frac{4}{5})$ and $(\frac{4}{5},\frac{1}{3})$ respectively.
The figure below shows an orbit that is not closed, with $(e,\beta)=(\frac{4}{5},\frac{2+\pi}{2\pi})$.
Thus, even if a potential can in principle accommodate closed orbits, the orbits need not always be closed. By Bertrand's theorem, only some potential can accommodate closed orbits.
The energy diagram will guarantee that you have bounded motion, but not necessarily closed orbits.