If we take "algebraic structure" to be a synonym for "algebra" (in the sense of universal algebra), then an algebraic structure is a set $X$, together with a family of operations on $X$.
Recall that given a set $X$, an "operation" on $X$ is a function $X^{\alpha}\to X$, where $\alpha$ is an ordinal. Such a function is called an $\alpha$-ary operation; when $\alpha$ is a natural number, the operation is said to be "finitary" (takes only finitely many arguments).
Sometimes, algebraic structures are further enriched with (i) "partial operations" (functions defined on a subset $A\subseteq X^{\alpha}$ rather than all of $X^{\alpha}$), or (ii) $\beta$-ary relations (subsetes of $A^{\beta}$). We can also impose identities (requires that the operations/relations satisfy certain properties such as commutativity, etc).
In this sense, vector spaces, groups, rings, fields, etc. are all (enriched) "algebras"; metric spaces are not.
"Space" is a bit fuzzier; I would not put "vector spaces" in the class, restricting it rather to things like topological spaces, manifolds, metric spaces, normed spaces, etc.
Now, one should realize that you this does not have to be a dichotomy: you can have structures that include both kinds of data: a topological group is both an algebraic structure (a group) and a space (topological space), in a way that makes both structures interact with one another "nicely". Normed vector spaces are both algebraic structures (vector spaces), and "spaces" (normed spaces, hence metric, hence topological), where, again, we ask that the two structures interact nicely.
In fact, there is a lot of interesting stuff that can be obtained by having the two kinds of structures and "playing them off against one another." For example, Stone Duality and Priestley Duality exploit this kind of "structured topological space" (a topological space that also has operations, partial operations, and relations that interact well with the topology).
We wouldn't say "closed-form" for the solution of a differential equation, neither "analytic expression" for the solution of a difference equation (at least this is my impression). On the other hand, most probably, in this particular example we would use them interchanged!
But yes, I would say that they are essentially interchangeable.
I would use also "explicit form" (again essentially interchangeable with those that you mention).
Best Answer
"Polynomial" is a precisely defined term. A polynomial is constructed from constants and variables by adding and multiplying. One could add "subtracting", but $x-y$ is $x+(-1)y$, so adding and multiplying are enough.
"Algebraic expression" is not a precisely defined term. Algebraic expressions include many things that are not polynomials, including rational funtions, which come from dividing polynomials, and things like $\sqrt{x}$.