In the structure theory of an operator acting on an infinite-dimensional vector space over a field $F$, there is some good news and a lot of bad news. The best news (as Matthew Emerton mentions) is the classification of finitely generated modules over a PID. This beautiful theorem immediately generalizes the Jordan canonical form theorem and the classification of finite or finitely generated abelian groups. It says that every f.g. module over a PID $R$ is a direct sum of cyclic modules $R/d$, and the summands are unique in either the primary decomposition (which comes from the proof by isotypic submodules and Jordan blocks) or the invariant factor decomposition (which comes from the Smith normal form proof). You can take $R = \mathbb{Z}$, or $R = F[x]$ for a field $F$, or of course there are other choices.
I suspect that the bad news is likely to be uniformly bad for countably generated modules over any PID. I found a reference for the case of torsion-free abelian groups, so I will concentrate on that case first. Consider the abelian groups $A$ that lie between $\mathbb{Z}^2$ and $\mathbb{Z}[\frac1p]^2$ for a prime $p$. Those choices for $A$ that are free abelian are just lattices in the plane (whose coordinates have denominators that are powers of $p$). There is an important theorem from the theory of affine buildings that you can make an infinite tree of valence $p+1$ out of these lattices. Two lattices are equivalent if they are the same lattice up to homothetic expansion (by a power of $p$). Two lattices are connected by an edge if they are nested and their quotient is $\mathbb{Z}/p$. So, the theorem says that this graph whose vertices are lattices is acyclic and has degree $p+1$ everywhere; the graph is an affine Bruhat-Tits building of type $A_1$.
Given a general $\mathbb{Z}^2 \subseteq A \subseteq \mathbb{Z}[\frac1p]^2$, you can let $A_n = (\frac1{p^n}\mathbb{Z}^2) \cap A$. This is the type of direct limit that Matthew Emerton describes, which looks good, but bad news is coming. For a while, $A_n/A_{n-1}$ can be $(\mathbb{Z}/p)^2$, then it can be $\mathbb{Z}/p$, and then $A_n$ might stop growing. If it stops growing, then $A$ is free with two generators. If it always grows by $(\mathbb{Z}/p)^2$, then $A = \mathbb{Z}[\frac1p]^2$. But in the middle case, if it grows by $\mathbb{Z}/p$, then $A$ is described by a path in the affine building from the origin to infinity. There are uncountably many ($2^{\aleph_0}$) such paths, and they have a $p$-adic topology.
So when are two such modules $\mathbb{Z}^2 \subseteq A \subseteq \mathbb{Z}[\frac1p]^2$ and $\mathbb{Z}^2 \subseteq B \subseteq \mathbb{Z}[\frac1p]^2$ isomorphic? The outer module is obtained by tensoring either $A$ or $B$ with $\mathbb{Z}[\frac1p]$, so the question is when $A$ and $B$ are equivalent under the action of $\text{GL}(2,\mathbb{Z}[\frac1p])$, which is a countable group. After localizing at all of the other primes, this question was studied by Hjorth, Thomas, and other logicians and algebraists. As Brian Conrad suggests, they have several theorems that indicate that the answer is wild, similar in spirit to the classification of infinite graphs up to isomorphism.
Now suppose that $F$ is a field, most conveniently a countable or finite field, and suppose that $V$ is an $F[x]$-module lying between $F[x]^2$ and $F[x,x^{-1}]^2$. Then you get the same geometric picture as before: The $F[x]$-free choices for $V$, up to homothety by powers of $x$, form an affine building which is an infinite tree, such that the neighbors of each vertex are a projective line $FP^1$. The complicated submodules are the ones that correspond to paths in this tree from the origin to infinity. If $F$ is finite or countable, then $\text{GL}(2,F[x,x^{-1}])$ is also countable, but the set of these paths is always uncountable and the group action is always far from transitive. I don't know if Hjorth's and Thomas' results extend to this case (because I am not qualified to read their papers), but my guess is that they would be comparably pessimistic.
There are other cases in which you get good news, basically by making infinite-dimensional operators look like finite-dimensional operators. The module $V$ in the previous paragraph has no $x$-eigenvectors even after passing to the algebraic closure $\bar{F}$. This is the same statement as that $V$ is torsion-free over $F[x]$. I think that a torsion module over a PID $R$ behaves much better, if it is the direct sum of its isotypic summands. For instance, if $x$ is locally nilpotent on $V$, then I think that it is a direct sum of Jordan blocks, possibly including infinite Jordan blocks (like the differentiation operator $x = \partial/\partial t$ on $\mathbb{Q}[t]$).
[First proof] I will address only the first part. Suppose $G$ is a compact group and $\pi : G \rightarrow U(H)$ a unitary irreducible representation on a Hilbert space $H$. Suppose $\phi$ a continuous function on $G$. Then it is easy to show that $\pi (\phi)$ is a compact operator on $H$.
Since the OP has asked for clarification let me state: convolution by continuous functions on $L^2(G)$ where $G$ a COMPACT group, is a compact operator. This fact is used in all proofs of Peter-Weyl. The proof is by showing that the convolution is an $L^2$ kernel, and any $L^2$ function on $G\times G$ may be approximated in $L^2$ by simple functions (linear combination of char functions of the form $E\times F$ where $E,F$ ae measurable subsets of $G$), each of these simple kernels have finite dimensional image and are hence compact operators. This is standard material in any functional analysis book (in fact Kirillov's book).
What I have used is that for a compact group, any irreducible unitary representation is a sub of the regular representation (see the comments), and hence $\pi (\phi)$ ,which is a convolution by $\phi$ is a compact operator.
If $(\phi _{\epsilon} )$ is an approximate identity on $G$ consisting of continuous conjugation invariant real valued continuous functions on $G$, then $\pi (\phi _{\epsilon})$ is a sequence of compact operators (which are scalars because of irreducibility) converging weakly to the identity, and hence the identity operator is also compact. Therefore, $H$ is finite dimensional.
I mention this because this does not use the Peter-Weyl theorem (but uses ideas of the proof).
[Second Proof] If you use the Peter-Weyl theorem, then the proof of finite dimensionality is easy. Peter weyl says that $L^2(G)$ is a Hilbert space direct sum of irreducible FINITE dimensional representations of $G$. The algebraic direct sum $X$ is then a dense subspace of $L^2(G)$. If an infinite dimensional irrep $H$ existed, it would still be in $L^2(G)$ but by Schur orthogonality (another use of Schur's lemma) functions in $H$ would be orthogonal to functions in $X$, a contradiction, since $X$ is dense.
[Remark] if $G$ is already a closed subgroup of $U(n)$, and $V$ is the standard rep of $U(n)$ then by the Stone-Weierstrass theorem, $V$, $V^*$ and irreducible subs of their tensor powers, will all be in $L^2(G)$ and will be dense in the space of continuous functions on $G$. And hence these representation functions $Y$ will be dense in $L^2(G)$. So vectors in $H$ cannot be orthogonal to the elements of $Y$. All this is standard material in any book on the Peter-Weyl theorem (actually worked out in Kirillov's book) and I suggest that the OP looks at them
[Remark] The proof given by Francois Ziegler is the simplest, in my opinion. .
Best Answer
If $A$ is finitely generated, as in your first edit, then this is much more elementary than the result of Dixmier that Faisal mentioned. A simple $A$-module would have the structure of a field extension of $k$ that is finitely generated as a $k$-algebra (since it's a quotient of $A$), but if $k$ is algebraically closed then there are no such extensions.