I would first like to make sure of the following concepts, note that all groups are abelian. Let $A,B$ be subgroups of $G$.
(1) Cartesian product (a.k.a. external product): The Cartesian product of subgroups $A$ and $B$ is $A \times B = \left\{ (a,b) \:{:}\: a \in A, b \in B \right\}$ where the operation by components is the group operation.
(2) Internal product: The internal product of subgroups $A$ and $B$ is $AB = \left\{ ab \:{:}\: a \in A, b \in B \right\}$.
(3) Sum: The sum of subgroups A and B is $A + B = \left\{ a + b \:{:}\: a \in A, b \in B \right\}$.
So (2) and (3) are basically the same objects because (2) is used under the multiplicative notation and (3) is used under the additive notation.
(4) Direct product: The internal product $AB$ is called a direct product and denoted $A \times B$ (confusingly enough) if $A \cap B = \left\{ 1 \right\}$.
(5) Direct sum: The sum $A + B$ is called a direct sum and denoted $A \oplus B$ if $A \cap B = \left\{ 0 \right\}$.
So (4) and (5) are basically the same objects because (4) is used under the multiplicative notation and (5) is used under the additive notation.
I am aware that (1) and (4) (or (5)) are isomorphic and often used interchangeably. When the direct sum is used in a correct way, it causes no confusion at all such as $\mathbb{Z}_{15} = \left\{ 0,5,10 \right\} \oplus \left\{ 0,3,6,9,12 \right\}$.
But when it is used in a rather ambiguous way such as $\mathbb{Z}_2 \oplus \mathbb{Z}_2$, which should be $\left\{ 0+0,0+1,1+0,1+1 \right\} = \left\{ 0,1 \right\}$, which is of order $2$, but I have seen my lecturer saying it is of order $4$, so he must have interpreted it as the Cartesian product owing to the isomorphic nature.
I am really not happy with this as the definition of sum and direct sum clearly states otherwise. Could anyone shed some enlightening lights on this issue?
Best Answer
Table of contents:
Let $\{G_i\}_{i\in I}$ be a family of groups.
The direct product is defined as
$$\prod_{i\in I}G_i = \{f\colon I\to \bigcup_{i\in I}G_i\mid f(i)\in G_i,\ \forall i\in I\}$$
with multiplication defined as $(f\cdot g)(i) = f(i)g(i)$. When $I$ is finite set, it is customary to write elements of direct product as $n$-tuples. For example, if $I=\{1,2\}$, we have
$$G_1\times G_2 = \{ (g_1,g_2)\mid g_i\in G_i,\ i = 1,2\}$$ and this relates to the above definition as ordered pair $(g_1,g_2)$ can be thought of as a function $$g\colon\{1,2\}\to G_1\cup G_2,\ g(1) = g_1,\, g(2) = g_2$$ and given another ordered pair $(g'_1,g'_2)$ and corresponding function $g'$, we can see that ordered pair $(g_1g_1',g_2g_2')$ corresponds to function $g\cdot g'$, so everything checks out.
Even in infinite case, we usually don't write elements of direct product as functions, we write them the same way we would write sequences, i.e. $(g_i)_{i\in I}$ and when $I$ is clear, we usually drop it in notation and just write $(g_i)$. We would write multiplication as $(g_i)\cdot (g'_i) = (g_ig_i')$.
Direct product is naturally equipped with family of projection homomorphisms $$\{\pi_j\colon\prod_{i\in I}G_i\to G_j\}_{j\in I}$$ defined by $\pi_j(f) = f(j)$, or $\pi_j((g_i)_{i\in I}) = g_j$.
Let $\{G_i\}_{i\in I}$ be a family of groups.
The direct sum is defined as
$$\bigoplus_{i\in I}G_i = \{f\colon I\to \bigcup_{i\in I}G_i\mid f(i)\in G_i,\ \forall i\in I,\ f(i)\neq e_i\ \text{only for finitely many}\ i\in I\}$$
with multiplication defined as $(f\cdot g)(i) = f(i)g(i)$. As you can see, definition is very similar to that of the direct product, however, elements of the direct sum are non-trivial only on finitely many places.
It is immediate that when $I$ is finite, direct product and direct sum coincide, but in infinite case, direct sum is proper subgroup of direct product. Notational conventions are the same as with direct product.
Direct sum is naturally equipped with family of inclusion homomorphisms $$\{\iota_j\colon G_j\to \bigoplus_{i\in I}G_i\}_{j\in I}$$ defined by $\iota_j(x) = (g_i)_{i\in I}$, where $g_i = e_i$ for $i\neq j$ and $g_j = x$. This way we can think of all $G_j$'s as subgroups of the direct sum. We will use this later, we will identify $G_j$ with its image $\iota_j(G_j)$ and just say that $G_j$ is a subgroup of $\bigoplus_{i\in I}G_i$ when we really mean that $\iota_j(G_j)$ is a subgroup of $\bigoplus_{i\in I}G_i$. This is a common abuse of notation that one gets used to in time.
We will now assume that all the groups are abelian and switch to additive notation. Given $x\in\bigoplus_{i\in I}G_i$ and $x = (x_i)$, we can write $x = \sum_{i\in I}\iota_i(x_i)$. Note that this sum is well-defined even when $I$ is infinite because $\iota_i(x) = 0$ for all but finitely many $i\in I$, so what looks like infinite sum is really a finite one. This is a crucial difference between direct product and direct sum. There is much more to be said about this, but it wouldn't serve much purpose at this point. You will learn about it in due time. For now, just notice that when we work with vector spaces, this is similar to writing a vector as a linear combination of basis vectors.
When we have subgroups $A,B\leq G$, their internal sum $A+B$ is as you defined. But let me return to non-commutative case and instead of internal sum talk about internal product $AB$.
Here's the problem, in general, $AB$ is not a subgroup of $G$. The reason is simple, $(ab)(a'b')$ may not be an element of $AB$ because we don't know what to do with $ba'$. What we would like is to write $ba'$ in the form $a''b''$, and then we would have $(ab)(a'b') = (aa'')(b''b) \in AB$. Thus, if $AB = BA$, $AB$ is a subgroup of $G$. The converse is true as well, if $AB$ is a subgroup of $G$, then $AB = BA$ (can you prove it?)
Thus, let us work with the assumption that $AB = BA$ (note that this is trivially true in abelian case). The next question is if we can define a group homomorphism $AB\to A\times B$. The obvious candidate would be $ab\mapsto (a,b)$, but is this well defined? It turns out that assuming $A\cap B = \{e\}$ it is, because it gives us that $a_1b_1 = a_2b_2$ if and only if $a_1 = a_2$ and $b_1 = b_2$ (can you prove it?) and in that case $ab\mapsto (a,b)$ is a group isomorphism (can you prove it?)
Notice how all these considerations become trivial when you assume that $G$ is abelian.
However, we can also bypass all of this by not considering $A$ and $B$ as subgroups of $G$, but as subgroups of $A\times B$ (or $A\oplus B$, which is the same) via inclusions that I mentioned above, i.e. $A \equiv A\times\{e\}$ and $B\equiv \{e\}\times B$. Now we have that $AB = BA$ and $A\cap B = \{e\}$ inside $A\times B$, so we get that $AB = A\times B$. Again, we are identifying $A$ with $A\times \{e\}$ and $B$ with $\{e\}\times B$. Notice how we don't even have to have $A\cap B = \{e\}$ in the original group $G$, we could even have $A = B$, and still get that $A\times A$ is the internal product of $A\times\{e\}$ and $\{e\}\times A$. This is important to notice considering your examples.
So, what of your examples?
Your first example is $\mathbb Z_3\oplus \mathbb Z_5$. What you did here is exactly what I describe above, you identify $\mathbb Z_3$ with appropriate subgroup of $\mathbb Z_{15}$, and the same thing with $\mathbb Z_5$. However, there is important thing to notice here, that you might be missing: you are implicitly using the fact that $\mathbb Z_3\oplus \mathbb Z_5\cong \mathbb Z_{15}$ and this works only because $3$ and $5$ are relatively prime. Actually, what you've done is a disguised proof of $\mathbb Z_3\oplus \mathbb Z_5\cong \mathbb Z_{15}$, because when $\mathbb Z_3$ and $\mathbb Z_5$ are considered as subgroups of $\mathbb Z_{15}$, they do satisfy $\mathbb Z_3\cap \mathbb Z_5 = \{0\}$, $\mathbb Z_3+\mathbb Z_5 = \mathbb Z_5+\mathbb Z_3 = \mathbb Z_{15}$ and thus $\mathbb Z_{15} \cong \mathbb Z_3\oplus \mathbb Z_5$ by the above general discussion.
As you've seen yourself, it fails in your second example and this is because $\mathbb Z_2\oplus\mathbb Z_2\not\cong \mathbb Z_4$ (nor to $\mathbb Z_2$, for that matter). However, when you do this properly it works.
Let $A = B = \mathbb Z_2$ and identify $A$ with subgroup $A\oplus\{0\}\leq \mathbb Z_2\oplus \mathbb Z_2$ and $B$ with subgroup $\{0\}\oplus \mathbb Z_2\leq \mathbb Z_2\oplus \mathbb Z_2$. Explicitly, $A$ is identified with $\{(0,0),(1,0)\}$ and $B$ is identified with $\{(0,0),(0,1)\}$. Notice how as in general discussion we now have $A\cap B = \{(0,0)\}$ (with the identification in mind) and \begin{align}A+B &= \{(0,0)+(0,0), (1,0)+(0,0), (0,0)+(0,1), (1,0)+(0,1)\}\\ &= \{(0,0), (1,0), (0,1), (1,1)\} = \mathbb Z_2\oplus \mathbb Z_2.\end{align}
Also, what you noticed yourself, when $A$ and $B$ are not identified with appropriate subgroups of $\mathbb Z_2\oplus \mathbb Z_2$, but instead considered as subgroups of $\mathbb Z_2$, we don't have $A\cap B = \{0\}$ and instead we have $A + B = \mathbb Z_2 \not\cong \mathbb Z_2\oplus \mathbb Z_2$.