I'm not sure about the "real" origin, but I think that the main source for the use of $\bot$ is Gerhard Gentzen : INVESTIGATIONS INTO LOGICAL DEDUCTION (ed or. Untersuchungen uber das logische Schliessen, Mathematische Zeitschrift 39 (1935) 176-210, 405-431); see english reprint : Gerhard Gentzen, The collected papers (1969), page 70 :
Symbols for definite propositions: $\top$ ('the true proposition'), $\bot$ ('the false
proposition').
This definitions license the name falsum for $\bot$.
I think that the term contradiction is more apt to a "metalogical" usage, like tautology.
See Gentzen, page 78 :
$\mathfrak A$ and $\lnot \mathfrak A$ signifies a contradiction and as such cannot hold true (law of contradiction). This is formally expressed by the inference figure $\lnot$-E where $\bot$ designates 'the contradiction', 'the false'.
Thus, I agree with Zhen Lin's comment : $⊥$ [the falsum] is "the abstract contradiction".
Gentzen's introduction of $\bot$ as a primitive symbol allows him to define $\lnot$ through an abbreviation: $\lnot A$ stands for : $A \rightarrow \bot$.
Axioms are not "defined to be true"; I'm not even sure what that would mean. What they are is evaluated as true. Practically speaking all this means is that in the mathematical context at hand, you're allowed to jot them down at any time as the next line in your proof.
Definitions have virtually nothing to do with truth, but are instead shorthand for formulae or terms of the language. Using the language of set theory as my example, "$x\subset y$" is going to be an abbreviation for "$\forall z(z\in x\to z\in y)$". If you were to put these two expressions on either side of a biconditional symbol, it would of course be true, but not because we have assumed it to be true, but rather because when you have unpacked everything into the actual formal language of set theory (of which $\subset$ is not a part) you have simply put exactly the same formula on both sides; it is a logical truth of the form $\phi\iff\phi$.
Update: I realized this answer would be more complete if I addressed the example you show above with $0$, and addressed comments made below by @MauroALLEGRANZA.
Let's say I want to define $0$ as a shorthand for the unique $x$ such that $\forall n(x\neq S(n))$. What this is saying is that we can state a uniqueness condition, namely "$\forall y(y=x\Leftrightarrow \forall n(y\neq S(n)))$," and, moreover, $\forall y(y=0\Leftrightarrow \forall n(y\neq S(n)))$. This latter, however, is a substantial statement entailing the existence of a certain kind of object, and if we don't have $\forall n(0\neq S(n))$ as an axiom, how will we derive it? It should be obvious that merely having a way to say "predecessor-less object" does nothing to guarantee the existence of a predecessor-less object; at best, you've shifted the burden of the axiom $\forall n(0\neq S(n))$ onto another axiom that circumlocutes the constant symbol $0$. Having two ways to say "predecessor-less object", one in the original language and one in a metalanguage, doesn't do any more work than only having one way to say it.
Sig. Allegranza brought up a variant where the defined symbol becomes a genuine formal symbol of an expanded formal language, and we only look at extensions of the theory axiomatizing the equivalence of the new predicate with a formula of the old language. In this case the axiom stating the equivalence is not even utterable in our old language, much less will it have any consequences for the models of said language. With our above example, we might have the new one-place predicate $Z(v)$ added to the language of $\mathsf{PA}$, and take as our new axiom $Z(v)\Leftrightarrow \forall n(v\neq S(n))$. That is, we have a predicate, now part of the formal language, that is equivalent to the statement that $v$ is predecessor-less. But now that there's a formal axiom about $Z$, just try to derive $\exists x\forall n(x\neq S(n))$, much less $\forall n(0\neq S(n))$, from just this axiom alone. It should be easy to see that you're not going to be able to derive any non-trivial sentences in the language of $\mathsf{PA}$.
In either case, we see that definitions simply don't do the work of axioms.
Best Answer
We wouldn't say "closed-form" for the solution of a differential equation, neither "analytic expression" for the solution of a difference equation (at least this is my impression). On the other hand, most probably, in this particular example we would use them interchanged!
But yes, I would say that they are essentially interchangeable. I would use also "explicit form" (again essentially interchangeable with those that you mention).