According to Joyal, Street ("An Introduction to Tannaka Duality and Quantum Groups"), any $k$-linear abelian category $\mathcal{C}$ admitting a faithful, exact functor $U: \mathcal{C} \rightarrow \mathcal{V}ect_{k}$ into finite-dimensional $k$-vector spaces arises as a category of finite-dimensional comodules over some coalgebra. In fact, the coalgebra in question is the "predual" $End^{pre}(U)$ of the endomorphism algebra of $U$.
Do all such abelian categories also arise as categories of finite-dimensional modules over some algebra? If not, what's the easiest example of a $k$-linear abelian category admitting a faithful, exact functor into finite-dimensional vector spaces which is not equivalent to a category of modules?
Edit: As pointed out below, one should also assume that $\mathcal{C}$ is essentially small so that the endomorphism coalgebra is a well-defined.
Best Answer
If $A$ is a $k$-algebra, and $M$,$N$ are finite-dimensional $A$-modules, then $$\operatorname{Ext}^i_A(M,N)\cong\operatorname{Tor}^A_i(M,N^*)^*$$ (where $*$ denotes $k$-dual).
So $\operatorname{Ext}^i_A(M,N)$ must be the dual of a vector space, and so in particular its dimension can't be countably infinite.
For $i=1$ it makes no difference whether we take $\operatorname{Ext}^1(M,N)$ in the category of finite-dimensional modules or the category of all modules, as it can be described in terms of equivalence classes of extensions $0\to N\to L\to M\to0$.
Let $\mathcal{C}$ be the category whose objects are finite-dimensional vector spaces over $k$ with a set of endomorphisms, all but finitely many zero, indexed by a countable set. Let $k$ be the one-dimensional object with all the endomorphisms zero. Then $\operatorname{Ext}^1(k,k)$ has countably infinite dimension, and so $\mathcal{C}$ can't be equivalent to the category of finite-dimensional modules for a $k$-algebra.
I think that if you translate this example into a question about profinite $k$-algebras as in Qiaochu's answer, it comes down to the fact that a countable product of copies of $k$ is not the profinite completion of any vector space over $k$.
Slightly changing the example (requiring the composition of any two of the endomorphisms to be zero) gives a simple example of a profinite $k$-algebra that is not a profinite completion:
Let $V$ be the direct product of countably many copies of $k$ with the product topology, and let $A=k\oplus V$ where $uv=0$ for all elements $u,v$ of $V$. The subalgebras of $A$ are all of the form $k\oplus U$ for some subspace $U\leq V$. The proper ideals of $k\oplus U$ are of the form $0\oplus U'$ for subspaces $U'\leq U$, so the profinite completion, as an algebra, of $k\oplus U$ is $k\oplus\hat{U}$, where $\hat{U}$ is the profinite completion, as a vector space, of $U$. But the profinite completion of any vector space $U$ is its double dual, which is larger than $V$ for any infinite dimensional $U$. So $A$ is not the profinite completion of any subalgebra.