One way to explain the connection is via the Hodge decomposition of the (singular) cohomology groups of an algebraic variety.
On one hand, viewing $X$ as a two-dimensional real manifold, and using say Mayer--Vietoris, one proves that $H_1(X,\mathbf{Z}) \cong \mathbf{Z}^{2g}$, and hence by duality, $H^1(X,\mathbf{C}) \cong \mathbf{C}^{2g}$.
On the other hand, let's upgrade Riemann's theorem to the Riemann--Roch theorem, which says that if $K$ is the canonical bundle (i.e. the bundle of holomorphic 1-forms) on $X$, then
$$l(D)-l(K-D) = \mathrm{ deg}\, D + 1 -\gamma.$$
(I changed $g$ to $\gamma$ because a priori this isn't the same as the previous $g$; indeed, that's what we want to prove.) Applying this with $D=0$ we get
$$1-l(K) = 1-\gamma$$
hence $l(K)=\gamma$. Now $l(K)$ by definition means the dimension of the space of global sections of the bundle $K$ of holomorphic 1-forms on $X$. Another notation for the same space is $H^0(X,\Omega^1)$, so we have $\mathrm{dim} \, H^0(X,\Omega^1)=\gamma$.
The statement of Hodge decomposition for curves is the following:
\begin{align}
H^1(X,\mathbf{C}) = H^0(X,\Omega^1) \oplus H^1(X,\mathcal{O}_X) \end{align}
and moreover the two direct summands on the right-hand side are conjugate, hence have the same dimension $\gamma$. (Don't worry about what $H^1(\mathcal{O}_X)$ actually is if you don't know the definition of sheaf cohomology; all that matters is that it has dimension $\gamma$.)
Now we know the dimensions of both sides of the displayed formula from 1. and 2. above, and plugging them in we get
$$2g=\gamma+\gamma;$$
that is, $g=\gamma$!
Edit: Maybe it's worthwhile mentioning a simpler proof for plane curves of the equality of these two numbers.
First of all, it's easy to deduce from the Riemann–Roch theorem that the degree of the canonical line bundle $K$ is
$$\text{deg} \, K = 2\gamma - 2.$$
On the other hand, if $X$ is a smooth curve of degree $d$ in $\mathbf{P}^2$, one can write down an explicit nonzero regular differential $\omega$ on $X$, and observe that it has $d(d-3)$ zeroes. Since $\omega$ is a section of $K$, this shows that
$$\text{deg} \, K = d(d-3).$$
Comparing these two expressions for the degree, we find that $\gamma=\frac12 (d-1)(d-2)$.
So our goal is now to show that the topological genus $g(X)$ is also equal to $\frac12 (d-1)(d-2)$. To do this, we use the fact that $X$ is a degree-$d$ branched cover of $\mathbf P^1$ with $d(d-1)$ ramification points (to see this, project from a general point in $\mathbf P^2$). This gives the formula
$$\chi(X) = 2d -d(d-1);$$
since $\chi(X)=2-2g$, this simplifies to give $g=\frac12 (d-1)(d-2)$, as required.
In my answer below, I was implicitly assuming that the covering on the boundary was trivial. As Mike Miller points out below, this is not necessarily the case. See his comments for more details.
Denote the closed orientable surface of genus $g$ with $b$ boundary components by $\Sigma_{g, b}$, and the closed non-orientable surface of genus $g$ with $b$ boundary components by $S_{g,b}$. Recall that $\chi(\Sigma_{g,b}) = 2 - 2g - b$ and $\chi(S_{g,b}) = 2 - g - b$.
If $p : M \to N$ is a covering map between manifolds with boundary, then it restricts to a covering map $p|_{\partial M} : \partial M \to \partial N$ of the same degree. So if $p : \Sigma_{g,b} \to \Sigma_{g',b'}$ is a degree $k$ covering map, then $b = kb'$. Moreover,
\begin{align*}
\chi(\Sigma_{g,b}) &= k\chi(\Sigma_{g',b'})\\
2 - 2g - b &= k(2 - 2g' - b')\\
2 - 2g - kb' &= k(2 - 2g' - b')\\
2 - 2g &= k(2 - 2g')\\
\chi(\Sigma_g) &= k\chi(\Sigma_{g'}).
\end{align*}
The converse is also true. That is, if $\chi(\Sigma_g) = k\chi(\Sigma_{g'})$ and $b = kb'$, then there is a degree $k$ covering map $\Sigma_{g,b} \to \Sigma_{g',b'}$. To see this, note that if $\chi(\Sigma_g) = k\chi(\Sigma_{g'})$, then there is a degree $k$ covering map $p : \Sigma_g \to \Sigma_{g'}$; see this answer. If $D \subset \Sigma_{g'}$ is the interior of a closed disc in $\Sigma_{g'}$, then $p^{-1}(D)$ is a disjoint union of the interiors of $k$ disjoint closed discs in $\Sigma_g$. So if $D_1, \dots, D_{b'}$ are the interiors of $b'$ disjoint closed discs in $\Sigma_{g'}$, then $p^{-1}(\Sigma_{g'}\setminus(D_1\cup\dots\cup D_{b'})$ is $\Sigma_g$ with $kb' = b$ interiors of disjoint closed discs removed, i.e. $\Sigma_{g,b}$. Therefore the restriction of $p$ to $\Sigma_{g,b}$ is a degree $k$ covering $\Sigma_{g,b} \to \Sigma_{g',b'}$.
Likewise, $\Sigma_{g,b}$ is a $k$-sheeted covering of $S_{g',b'}$ if and only if $\chi(\Sigma_g) = k\chi(S_g)$, $b = kb'$ and $k$ is even. Note that $k$ must be even as any orientable covering of a non-orientable manifold must factor through the orientation double cover.
Therefore, we have the following complete list of coverings:
- $\Sigma_{5,4} \to \Sigma_{5,4}$ of degree one,
- $\Sigma_{5,4} \to \Sigma_{3,2}$ of degree two,
- $\Sigma_{5,4} \to S_{6,2}$ of degree two,
- $\Sigma_{5,4} \to \Sigma_{2,1}$ of degree four, and
- $\Sigma_{5,4} \to S_{4,1}$ of degree four.
Best Answer
As a smooth manifold, it's homeomorphic to $S^1 \times (-1, 1)$, with one homeomorphism being $$ (\theta, u) \mapsto \left(\cos \theta, \sin \theta, \tan \left(\frac{\pi}{2} u \right) \right). $$
It's also homotopy-equivalent to $S^1$, via the deformation-retraction
$$ H(\theta, t; s) = (\theta, st) $$ from $S^1 \times (-1, 1)$ onto the subset $S^1 \times \{0\}$.
The key points of these two observation are
"not closed because it's infinitely long" really isn't right -- it's not closed because it's not closed, and this can happen in many ways.
As for "topology" of the cylinder, there are several notions of equivalence in topology, with homeomorphism at one end, and homotopy equivalence being somewhat weaker. Since your shape is homotopy equivalent to a circle, but you regard your shape as "not closed" and the circle as "closed", you probably don't want to use the notion of homotopy equivalence -- it's too weak for your purposes.