Let me elaborate on Tsemo Aristide's anwser that the direct sum of abelian groups should be replaced by the so-called free product for non-abelian groups.
When we want to compare a construction from the theory of abelian groups with the situation of general groups, we first need a way of comparing the class of abelian groups with the class of groups in general. The usual way of doing this is to use categories. A category is just a collection of objects, together with special 'maps' between them (note that I'm glossing over a whole lot of details here, but the Wikipedia page on categories has a lot of information). For instance, there is the category of groups (with group homomorphisms as maps), the category of $\mathbb{R}$-vector spaces (with linear maps), the category of sets (with no restriction on the maps), etc.
In our case, we want to compare the category of groups with the category of abelian groups, and somehow transfer the notion of a direct sum from the latter to the former. To do that, we first need to express the direct sum in the language of categories, i.e. just refering to objects (abelian groups) and maps (group homomorphisms), and in particular without refering to elements of the groups involved. That may seem close to impossible at first glance, but it can be done using the following trick.
First, we notice that if $(A_i)_{i\in I}$ is any collection of abeliqn groups, then for any other abelian group $B$ there is a very natural (set-theoretic) bijection
$$\textrm{Hom}(\oplus_{i \in I} A_i, B) \to \prod_{i \in I}\textrm{Hom}(A_i, B),$$
where $\oplus_{i \in I} A_i$ is the direct sum of the $A_i$ (you should at this point take the time to write down which map this is, and convince yourself that this map is indeed bijective). In words, this says that a homomorphism from $\oplus_{i \in I} A_i$ to $B$ and a collection of morphisms from every $A_i$ to $B$ are 'the same thing'. In fact, it turns out that the direct sum is unique with this property: if $A$ is any abelian group such that for any abelian group $B$ we get a natural bijection like above, then $A$ is isomorphic to $\oplus_{i \in I} A_i$. So this gives us precisely a characterisation of the direct product in terms of just the groups and the homomorphisms.
Now that we know what a direct sum looks like in the category of abelian groups, we can try doing the same for general groups. So given a collection $(G_i)_{i \in I}$ of not necessarily abelian groups, we want to find a group $G$ with the property that for any group $H$, there is a natural bijection
$$\textrm{Hom}(G, H) \to \prod_{i \in I}\textrm{Hom}(G_i, H).$$
Based on the abelian case, we might expect that this group $G$ should be the direct product of the $G_i$, or perhaps the subset of the direct product $G_i$ of sequences with only finitely many non-identity components, but unfortunately these groups do not satisfy the above condition (you should verify this by finding some explicit examples of sets $I$ and groups $G_i$ and $H$ for which we do not have a bijection like above). The group that does satisfy the condition is the one called the free product of the $G_i$. This group is formed by considering finite words, where the letters are taken from all the sets $G_i$, and we may simplify words by multiplying two adjecent letters if they come from the same $G_i$, and we may remove any identity elements from the words (see the wiki page for a more precise definition). It is a nice exercise to show that in the case that all the $G_i$ are abelian, the direct sum of the $G_i$ is isomorphic to the abelianization of the free product, so the free product is indeed a generalization of the direct sum.
So we see that the 'correct' translation of the direct sum concept to general groups leads to free products, instead of cartesian products or subsets thereof. In other words, there is not a problem with considering 'direct sums' of non-abelian groups per se, but to preserve the properties that the direct sum has in the abelian case, we need to consider the free product instead of a direct sum.
Best Answer
Like Qiaochu says, the key here is that the direct product of finitely many abelian groups functions as both the (categorical) product and the (categorical) coproduct in the category of abelian groups.
And before your eyes glaze over, what that means is that:
A homomorphism from an abelian group $A$ into a (finite) direct product $G_1\times G_2\times\cdots\times G_n$ of abelian groups is equivalent to a family $f_1,\ldots,f_n$ of homomorphisms $f_i\colon A\to G_i$; (in fact, this holds for arbitrary groups, and arbitrarily many direct factors, not just finitely many); and
A homomorphism from a finite direct product $G_1\times G_2\times\cdots\times G_n$ of abelian groups into an abelian group $B$ is equivalent to a family $g_1,\ldots,g_n$ of homomorphisms $g_i\colon G_i\to B$ (here we do need both finiteness and abelianness).
The first equivalence is easy: given a map $f\colon A\to G_1\times\cdots\times G_n$, the maps $f_i$ are just the compositions of $f$ with the canonical projections $\pi_i\colon G_1\times\cdots\times G_n\to G_i$; going the other way, given a family $f_1,\ldots,f_n$ of maps, you get the map $A\to G_1\times\cdots\times G_n$ by $f(a) = (f_1(a),f_2(a),\ldots,f_n(a))$.
For the second equivalence, given a homomorphism $g\colon G_1\times\cdots\times G_n\to B$, we define the maps $g_i\colon G_i\to B$ by restricting $g$ to the subgroup $\{0\}\times\cdots \times \{0\}\times B_i\times\{0\}\times\cdots\times\{0\}$. Conversely, given a family of homomorphisms $g_1,\ldots,g_n$, we construct the map $g$ by $g(x_1,\ldots,x_n) = g_1(x_1)+g_2(x_2)+\cdots+g_n(x_n)$; here, both the fact that the product has only finitely many factors and that the groups are abelian is important.
Now let $A=B=G_1\times\cdots\times G_n$. Then a homomorphism from $A$ to $B$ is equivalent, by 1, to a family of homomorphisms $\Phi_j\colon A\to G_j$. And by 2, each $\Phi_j$ is equivalent to a family of homomorphisms $\phi_{ij}\colon G_i\to G_j$. Thus, each homomorphism from $A$ to $B$ is equivalent to a family $\{\phi_{ij}\mid 1\leq i,j\leq n\}$, with $\phi_{ij}\colon G_i\to G_j$.
Now suppose you have two homomorphisms, $\Phi,\Psi\colon A\to B$, and you want to compose them. If $\Phi$ corresponds to $\{\phi_{ij}\}$ and $\Psi$ corresponds to $\{\psi_{ij}\}$, what does the composition correspond to in terms of maps $G_i\to G_j$?
If you trace the correspondence carefully, you should find that the induced map from $G_i$ to $G_j$ is precisely $$\psi_{i1}\circ\phi_{1j} + \psi_{i2}\circ\phi_{2j}+\cdots+\psi_{in}\circ\phi_{nj},$$ so that if you arrange the families $\{\phi_{ij}\}$ and $\{\psi_{ij}\}$ into matrices, composition corresponds to matrix multiplication in the usual way (though because composition is not commutative, you have to be mindful of the order of the products.
Once you have that endomorphisms can be "coded" as matrices with composition corresponding to matrix multiplication, the fact that automorphisms correspond to invertible matrices follows immediately. However, actually writing down a formula is complicated, because these matrices have entries that don't commute with one another; even in simple cases, like trying to do something like $C_{p^{\alpha}}\times C_{p^{\beta}}$ with $\alpha\gt\beta$, writing down the inverse of an automorphism in terms of its entries turns into a computation with congruences that is difficult to write down as a formula. But never fret, you aren't asked for an explicit formula.