Suppose that I can find a subset of C: $C_s \subset C$, which has a finite number of elements: $C_s = \{c_{s_1}, \ldots, c_{s_n}\}$, so that every element of $C$ can be expressed as a "sum" of the elements of $C_s$.
Making "sum" precise here would be to say that every $c\in C$ can be obtained from the elements of $C_s$ by finitely many $+$ operations. You would then call $C_s$ a generating set of $(C,+)$
- Is $C_s$ necessarily closed under +?
No it is not, consider the magma $(\mathbb Z_{>0}, +)$ with generating set $\{1\}$.
- Since my $+$ is not associative or commutative, I could potentially have an infinite number of operands correct?
No, you can only have finitely many operands. Just like in any structure without a notion of limits, you only have finite products, finite sums, finite linear combinations, etc.
Of course, you can have arbitrarily many operands in a finite (non-associative) sum.
2.1 subsidiary question: is there a clean/short way to write a non-associative non-commutative binary operation on a possibly infinite number of elements? Something like $\sum_k^n ...$ but which clearly shows that there can be repeating elements (like $c_{s_4}$ above) and that the "sum" is not commutative/associative?
You can draw finite binary rooted trees with leaves labelled by elements of the magma. You have to encode the order of operations (the brackets) one way or another.
- How would I properly define this "linear independence" property in my magma with a non-associative and non-commutative +?
The closest you can get is saying a subset $I\subseteq C$ is independent if every $c\in I$ can not be generated from $I\setminus\{c\}$. This is an analogy with a subset of a vector space being linearly independent iff no vector can be written as a linear combination of the others.
You could then define a "basis" to be an independent generating set, but do not expect any nice properties from this.
This answer contains three separate ways to do this.
First: a partial magma could be defined as a set $Y$, together with a function $m\colon Y\times Y \to Y + 1$ representing the binary operation, where $1$ is the one-element set and $+$ is the coproduct of sets (disjoint union).
This suggests that a natural notion of morphism between partial magmas $(Y,m)$ and $(Y',m')$ would be a function $f\colon Y\to Y'$ such that $f(m(x,y)) = m'(f(x),f(y))$, for all $x,y\in Y$, as you suggested in the comments.
Another option, which I think is more natural, is to define it as a magma in the category of partial maps. To define that category, take the objects as sets and let a morphism $f\colon X\to Y$ be any function $f\colon X\to Y+1$, so it either returns an element of $X$ or a special element meaning "undefined", which I'll write as $\bot$. Define composition as
$$(f;g)(x) = \begin{cases}
\bot &\text{if $f(x)=\bot$}\\
g(f(x)) &\text{otherwise.}
\end{cases}
$$
This is the Kleisli category of what Haskell people call the Maybe monad. We can make it into a symmetric monoidal category by letting $\otimes$ be the Cartesian product of sets on objects, and letting
$$
(f\otimes g)((x,y)) = \begin{cases}
\bot &\text{if $f(x)=\bot$ or $g(y)=\bot$}\\
(f(x),g(y)) &\text{otherwise.}
\end{cases}
$$
A magma in this category consists of a set $Y$, together with a partial map $m\colon Y\times Y\to Y$, so it's also a partial magma as you defined it. But now a magma homomorphism from $(Y,m)$ to $(Y',m')$ is a partial function $f\colon Y\to Y'$, such that $f(m(x,y)) = m'(f(x),f(y))$ for all $x,y\in Y$. The difference between this and the other construction above is that $f$ is a partial function, so that $m'(f(x),f(y))$ is allowed to be undefined even if $m(x,y)$ is defined.
As well as seeming more natural, this second solution is probably easier to work with. Being a magma in some category means that many of the things you can prove about magmas will also be true of partial magmas, since the proof will still work unless it relies on some specific property of $\mathsf{Set}$ that isn't shared by the category of partial maps.
It occurs to me that you might want something different to either of the above. You mentioned in a comment that you want to generalise categories, where you see composition as a partial magma on the set of morphisms. In that case it seems like you would want a functor to be a special case of a magma morphism. But functors behave kind of opposite to what I just described. You can have morphisms $f$ and $g$ that aren't composable in a
category $\mathscr{C}$, but their images under a functor $F\colon\mathscr{C}\to\mathscr{D}$ are composable in $\mathscr{D}$. But you can't have it the other way around - if two morphisms are not composable in $\mathscr{D}$ then they also have to be non-composable in $\mathscr{C}$.
So perhaps you want a morphism to be a function (not a partial function) $f\colon Y\times Y\to Y$, such that if $m'(f(x),f(y))=\bot$ then $f(m(x,y))=\bot$, otherwise $f(m(x,y))=m'(f(x),f(y))$.
I don't currently know a more abstract way to construct that, or have much intuition about its properties. I'll think about it.
Best Answer
Suppose $S$ is a subtractive commutative magma in which subtraction is commutative. Then for any $a,b\in S$, we have $$(a+b)+b=(a+b)+((a+b)-a)=(a+b)+(a-(a+b))=a$$ which means $a+b=a-b$. Conversely, if $S$ is a subtractive commutative magma in which $a+b=a-b$ for all $a$ and $b$, then subtraction is obviously commutative. So commutativity of subtraction is equivalent to subtraction being the same as addition. Or, a commutative magma has subtraction which is commutative iff it satisfies the identity $(a+b)+b=a$ (since this identity implies subtraction exists and coincides with addition).
In particular, if $S$ is a group, subtraction being the same as addition just means that $a=-a$ or $2a=0$ for all $a\in S$. Thus the abelian groups with commutative subtraction are exactly the abelian groups in which every element has order dividing $2$, i.e. the vector spaces over $\mathbb{Z}/2\mathbb{Z}$.
A bit more generally, suppose $S$ is a nonempty subtractive commutative semigroup. Fix some element $a\in S$ and let $0=a-a$. Then for any $b\in S$, $a+0+b=a+b$ and so $0+b=(a+b)-b=b$. That is, $0$ is an identity element. We also see that $0-b$ is an inverse for $b$ and so $S$ is a group. Thus the semigroup case reduces to the group case: a subtractive commutative semigroup with commutative subtraction is either empty or a vector space over $\mathbb{Z}/2\mathbb{Z}$.
For more general (not necessarily associative) magmas, I doubt there is any nice classification. Here is one neat class of examples. Let $(S,+)$ be any abelian group and fix an element $z\in S$. Define an operation $\oplus$ on $S$ by $$a\oplus b=z-a-b.$$ I claim $(S,\oplus)$ is a commutative subtractive magma in which subtraction is commutative. It is clear that $\oplus$ is commutative. To see it is subtractive and subtraction is commutative, note that $a\oplus c=b$ iff $c=z-a-b$ and so the difference between $a$ and $b$ with respect to $\oplus$ is $z-a-b$.
A bit more generally, you could also take any subset of $S$ closed under this operation $\oplus$. Note though that not every example arises in this way. Indeed, any example of this form satisfies the identity $$((a\oplus b)\oplus c)\oplus d=((a\oplus d)\oplus c)\oplus b$$ (they are both equal to $z-a-b-d+c$). However, this identity does not hold in every example; in particular, it does not hold in the free commutative magma satisfying the identity $(a+b)+b=a$ on four elements. (Proof sketch: say that a word is reduced if it cannot be shrunken using $(a+b)+b=a$. Prove by induction on length that each word can be reduced to a unique reduced word (modulo swapping the order of sums). Conclude that each element of a free object is represented by a unique reduced word on the generators. Thus if $a,b,c,d$ are free generators, then $((a+b)+c)+d$ and $((a+d)+c)+b$ are distinct reduced words and so are not equal.)