Table of contents:
- Direct products in general.
- Direct sums in general.
- Internal direct product (sum) in general.
- A look at your examples.
Let $\{G_i\}_{i\in I}$ be a family of groups.
The direct product is defined as
$$\prod_{i\in I}G_i = \{f\colon I\to \bigcup_{i\in I}G_i\mid f(i)\in G_i,\ \forall i\in I\}$$
with multiplication defined as $(f\cdot g)(i) = f(i)g(i)$. When $I$ is finite set, it is customary to write elements of direct product as $n$-tuples. For example, if $I=\{1,2\}$, we have
$$G_1\times G_2 = \{ (g_1,g_2)\mid g_i\in G_i,\ i = 1,2\}$$ and this relates to the above definition as ordered pair $(g_1,g_2)$ can be thought of as a function $$g\colon\{1,2\}\to G_1\cup G_2,\ g(1) = g_1,\, g(2) = g_2$$ and given another ordered pair $(g'_1,g'_2)$ and corresponding function $g'$, we can see that ordered pair $(g_1g_1',g_2g_2')$ corresponds to function $g\cdot g'$, so everything checks out.
Even in infinite case, we usually don't write elements of direct product as functions, we write them the same way we would write sequences, i.e. $(g_i)_{i\in I}$ and when $I$ is clear, we usually drop it in notation and just write $(g_i)$. We would write multiplication as $(g_i)\cdot (g'_i) = (g_ig_i')$.
Direct product is naturally equipped with family of projection homomorphisms $$\{\pi_j\colon\prod_{i\in I}G_i\to G_j\}_{j\in I}$$ defined by $\pi_j(f) = f(j)$, or $\pi_j((g_i)_{i\in I}) = g_j$.
Let $\{G_i\}_{i\in I}$ be a family of groups.
The direct sum is defined as
$$\bigoplus_{i\in I}G_i = \{f\colon I\to \bigcup_{i\in I}G_i\mid f(i)\in G_i,\ \forall i\in I,\ f(i)\neq e_i\ \text{only for finitely many}\ i\in I\}$$
with multiplication defined as $(f\cdot g)(i) = f(i)g(i)$. As you can see, definition is very similar to that of the direct product, however, elements of the direct sum are non-trivial only on finitely many places.
It is immediate that when $I$ is finite, direct product and direct sum coincide, but in infinite case, direct sum is proper subgroup of direct product. Notational conventions are the same as with direct product.
Direct sum is naturally equipped with family of inclusion homomorphisms $$\{\iota_j\colon G_j\to \bigoplus_{i\in I}G_i\}_{j\in I}$$ defined by $\iota_j(x) = (g_i)_{i\in I}$, where $g_i = e_i$ for $i\neq j$ and $g_j = x$. This way we can think of all $G_j$'s as subgroups of the direct sum. We will use this later, we will identify $G_j$ with its image $\iota_j(G_j)$ and just say that $G_j$ is a subgroup of $\bigoplus_{i\in I}G_i$ when we really mean that $\iota_j(G_j)$ is a subgroup of $\bigoplus_{i\in I}G_i$. This is a common abuse of notation that one gets used to in time.
We will now assume that all the groups are abelian and switch to additive notation. Given $x\in\bigoplus_{i\in I}G_i$ and $x = (x_i)$, we can write $x = \sum_{i\in I}\iota_i(x_i)$. Note that this sum is well-defined even when $I$ is infinite because $\iota_i(x) = 0$ for all but finitely many $i\in I$, so what looks like infinite sum is really a finite one. This is a crucial difference between direct product and direct sum. There is much more to be said about this, but it wouldn't serve much purpose at this point. You will learn about it in due time. For now, just notice that when we work with vector spaces, this is similar to writing a vector as a linear combination of basis vectors.
When we have subgroups $A,B\leq G$, their internal sum $A+B$ is as you defined. But let me return to non-commutative case and instead of internal sum talk about internal product $AB$.
Here's the problem, in general, $AB$ is not a subgroup of $G$. The reason is simple, $(ab)(a'b')$ may not be an element of $AB$ because we don't know what to do with $ba'$. What we would like is to write $ba'$ in the form $a''b''$, and then we would have $(ab)(a'b') = (aa'')(b''b) \in AB$. Thus, if $AB = BA$, $AB$ is a subgroup of $G$. The converse is true as well, if $AB$ is a subgroup of $G$, then $AB = BA$ (can you prove it?)
Thus, let us work with the assumption that $AB = BA$ (note that this is trivially true in abelian case). The next question is if we can define a group homomorphism $AB\to A\times B$. The obvious candidate would be $ab\mapsto (a,b)$, but is this well defined? It turns out that assuming $A\cap B = \{e\}$ it is, because it gives us that $a_1b_1 = a_2b_2$ if and only if $a_1 = a_2$ and $b_1 = b_2$ (can you prove it?) and in that case $ab\mapsto (a,b)$ is a group isomorphism (can you prove it?)
Notice how all these considerations become trivial when you assume that $G$ is abelian.
However, we can also bypass all of this by not considering $A$ and $B$ as subgroups of $G$, but as subgroups of $A\times B$ (or $A\oplus B$, which is the same) via inclusions that I mentioned above, i.e. $A \equiv A\times\{e\}$ and $B\equiv \{e\}\times B$. Now we have that $AB = BA$ and $A\cap B = \{e\}$ inside $A\times B$, so we get that $AB = A\times B$. Again, we are identifying $A$ with $A\times \{e\}$ and $B$ with $\{e\}\times B$. Notice how we don't even have to have $A\cap B = \{e\}$ in the original group $G$, we could even have $A = B$, and still get that $A\times A$ is the internal product of $A\times\{e\}$ and $\{e\}\times A$. This is important to notice considering your examples.
So, what of your examples?
Your first example is $\mathbb Z_3\oplus \mathbb Z_5$. What you did here is exactly what I describe above, you identify $\mathbb Z_3$ with appropriate subgroup of $\mathbb Z_{15}$, and the same thing with $\mathbb Z_5$. However, there is important thing to notice here, that you might be missing: you are implicitly using the fact that $\mathbb Z_3\oplus \mathbb Z_5\cong \mathbb Z_{15}$ and this works only because $3$ and $5$ are relatively prime. Actually, what you've done is a disguised proof of $\mathbb Z_3\oplus \mathbb Z_5\cong \mathbb Z_{15}$, because when $\mathbb Z_3$ and $\mathbb Z_5$ are considered as subgroups of $\mathbb Z_{15}$, they do satisfy $\mathbb Z_3\cap \mathbb Z_5 = \{0\}$, $\mathbb Z_3+\mathbb Z_5 = \mathbb Z_5+\mathbb Z_3 = \mathbb Z_{15}$ and thus $\mathbb Z_{15} \cong \mathbb Z_3\oplus \mathbb Z_5$ by the above general discussion.
As you've seen yourself, it fails in your second example and this is because $\mathbb Z_2\oplus\mathbb Z_2\not\cong \mathbb Z_4$ (nor to $\mathbb Z_2$, for that matter). However, when you do this properly it works.
Let $A = B = \mathbb Z_2$ and identify $A$ with subgroup $A\oplus\{0\}\leq \mathbb Z_2\oplus \mathbb Z_2$ and $B$ with subgroup $\{0\}\oplus \mathbb Z_2\leq \mathbb Z_2\oplus \mathbb Z_2$. Explicitly, $A$ is identified with $\{(0,0),(1,0)\}$ and $B$ is identified with $\{(0,0),(0,1)\}$. Notice how as in general discussion we now have $A\cap B = \{(0,0)\}$ (with the identification in mind) and \begin{align}A+B &= \{(0,0)+(0,0), (1,0)+(0,0), (0,0)+(0,1), (1,0)+(0,1)\}\\ &= \{(0,0), (1,0), (0,1), (1,1)\} = \mathbb Z_2\oplus \mathbb Z_2.\end{align}
Also, what you noticed yourself, when $A$ and $B$ are not identified with appropriate subgroups of $\mathbb Z_2\oplus \mathbb Z_2$, but instead considered as subgroups of $\mathbb Z_2$, we don't have $A\cap B = \{0\}$ and instead we have $A + B = \mathbb Z_2 \not\cong \mathbb Z_2\oplus \mathbb Z_2$.
Best Answer
I'm used to central products requiring isomorphic centers, and then asking that all the isomorphisms be compatible. So given a family $\{A_i\}_{i\in I}$, you would have isomorphisms $\theta_{ij}\colon Z(A_i)\to Z(A_j)$, and require $\theta_{ii}=\mathrm{id}_{Z(A_i)}$, $\theta_{ji}=\theta_{ij}^{-1}$, and $\theta_{ik}=\theta_{jk}\circ\theta_{ij}$ for all $i,j,k$. Then you would take the group $\prod_{i\in I}A_i/N$, where $N$ is generated by all elements identifying $a_i\in Z(A_i)$ with $\theta_{ij}(a_i)$ in $Z(A_j)$.
Alternatively, one could generalize the definition following the same lines as Hanna Neumann did for the generalized free product with amalgamation (Neumann, Hanna. Generalized Free Products with Amalgamated Subgroups. American Journal of Mathematics, vol. 70, no. 3, 1948, pp. 590–625. JSTOR link; and Generalized free products with amalgamated subgroups, Amer. J. Math. vol. 72, no. 3 (Jul 1949), pp. 491-540, JSTOR link ). Let $\{G_i\}_{i\in I}$ be a family of groups, and for each $i,j\in I$ let $A_{ij}\leq Z(G_i)$ be specified, with isomorphisms $\theta_{ij}\colon A_{ij}\to A_{ji}$ such that $\theta_{ij}=\theta_{ji}^{-1}$, and $\theta_{ii}=\mathrm{id}$. We say a group $G$ is the central product of the $G_i$ at the $A_{ij}$ if and only there exist embeddings $f_i\colon G_i\to G$ such that
A necessary condition for this to work is that if $i,j,k$ are three indices, then $f_{i}(A_{ij}\cap A_{ik})$ be identified with $f_j(A_{ji}\cap A_{jk})$, with $f_k(A_{ki}\cap A_{kj})$, and that the identification be such that $\theta_{ik}|_{A_{ij}\cap A_{ik}} = \theta_{jk}\circ\theta_{ij}|_{A_{ij}\cap A_{ik}}$.