First definition (algebraic): A monoid is a pair $(M,b)$, where $M$ is a set (called the underlying set of the monoid) and $b\colon M\times M\to M$ is a mapping (called the binary operation of the monoid; for $m_1,m_2\in M$ denote $b(m_1,m_2)=m_1\bullet m_2$), which satisfy the following two properties:
1). for any $m_1,m_2,m_3\in M$ the following equality holds (associativity):
$$(m_1\bullet m_2)\bullet m_3=m_1\bullet(m_2\bullet m_3).$$
2). there exists such $e\in M$ (called the identity of the monoid), that for any $m\in M$ the following equalities hold: $e\bullet m=m\bullet e=m$.
It is a standard definition of a monoid. There are a lot of examples of monoids; for example, any group is a monoid. However, there are monoids, which are not groups.
You are right, $(\mathbb{Z},+)$, $(\mathbb{Z},\cdot)$ are monoids, but $(\mathbb{Z},-)$ is not (because, for example, $1-(1-1)\ne(1-1)-1$).
If you are familiar with category theory, then you can get a lot of natural examples in different categories. The reason is following:
Let $A$ be a category, $a\in A$ be its object. Then any morphism $f\colon a\to a$ is called an endomorphism of $a$. Thus we can consider the set of all endomorphisms of the object $a$, denote it by $end(a)$. Note, that we can composite any two morphisms in $end(a)$, therefore we get a binary operation on $end(a)$!
$$
\bullet\colon end(a)\times end(a)\to end(a);\qquad f\bullet g=f\circ g.
$$
It is a monoid by the definition of category.
Now you can take any category, for example $\mathbf{Set}$, take any its object -- for example, $\mathbb{N}$ (which is considered without its algebraic structure), -- and you get the monoid $\mathbb{N}^{\mathbb{N}}$ of all functions $f\colon\mathbb{N}\to\mathbb{N}$, which, of course, is not a group (check it).
Now it is easy to believe in the following definition of a monoid:
Second definition (category-theoretic): Monoid is a category with one object.
Indeed, denote by $x$ its single object, then $end(x)$ is a corresponding monoid. Conversely, if you have a monoid $(M,b)$, you can define a category $\mathbf{M}$ with single object, such that $Arr(\mathbf{M})=M$, and composition defined by the following rule:
$$
\forall f,g\in M:\;f\circ g=f\bullet g.
$$
But it was, of course, intuitive reasoning. In order to make an exact statement, I have to give a few more definitions:
1). Let $(M,\bullet_M)$ and $(N,\bullet_N)$ be monoids. Monoid homomorphism is a mapping $f\colon M\to N$, such that for all $m_1,m_2\in M$ the equalities $f(m_1\bullet_M m_2)=f(m_1)\bullet_N f(m_2)$ and $f(e_M)=e_N$ hold. It's easy to check that all small monoids and their homomorphisms form a category, called $\mathbf{Mon}$.
2). Let's denote by $\mathbf{S}$ the full subcategory of $\mathbf{Cat}$, which objects are categories with fixed single object.
Now I can formulate the exact statement (equivalence of two definitions):
Statement: The category $\mathbf{Mon}$ of monoids is isomorphic to the category $\mathbf{S}$.
It doesn't make a difference what the object is, so I doubt anybody has ever specified what it is. This is the same kind of thing as when you consider a "one-point space" $\{*\}$ in topology.
You said that a morphism is determined by its domain and codomain. (Or source and target object, so to speak.) That's not true, even in the category of sets. There's usually more than one mapping from a set $A$ to a set $B$.
For the last thing, what you wrote is correct. Of course $1$ and $2$ are not true "mappings," but they're what stands in for mappings in this category. If $1$ and $2$ are thought of as analogous to mappings, then the operation $+$ should be thought of as analogous to $\circ$. So $1 \circ 2 = 3$, etc.
By the way, this category has a concrete realization as the category with a single object $\mathbf{Z}$ whose morphisms are all automorphisms of $\mathbf{Z}$ as an ordered set.
Best Answer
No, that's not the way to view it. In order to view a monoid as a category, you have a single object $\mathsf{Andreas}$, and each element of the monoid is one morphism $\mathsf{Andreas}\to\mathsf{Andreas}$ in the category. The monoid operation is the composition in the category.
So for the integers, you don't have a morphism "add 1", but a morphism that is simply called $1$. And composition in the category works such that $1$ composed with $1$ is the morphism called $2$.
This is an example of a category where the morphisms are not functions.