The category of complexes in an abelian category $\mathcal{A}$ is a full subcategory of $\text{Fun}({\mathbb{Z}},\mathcal{A})$, where $\mathbb{Z}$ is partially ordered under reverse inequality. So if we know how these constructions are performed in the category of functors from $\mathbb{Z}$ to $\mathcal{A}$, we'll have the natural candidates to the category of complexes. The standard result is that (co)limits in $\text{Fun}({\mathcal{D}},\mathcal{C})$, where $\mathcal{D}$ is a small category and $\mathcal{C}$ is a category, are computed pointwise. Take a look at Borceux's 'Handbook of Categorical Algebra, Vol.$1$' section $2.15$. There he explains the precise meaning of being computed pointwise.
Since any abelian category is finitely (co)complete, we can compute any finite (co)limit in $\text{Fun}({\mathbb{Z}},\mathcal{A})$ pointwise. If we consider a (co)complete category, e.g., the category of modules over a ring, we can compute any (co)limit pointwise.
If you think Borceux's book is too terse, there is a similar discussion in Rotman's 'An Introduction to Homological Algebra' on page $317$.
Added: In order to remove the iotas you will need to prove that $\alpha_{n-1}\partial_n^\oplus\iota_n^B=\partial_n^C\alpha_n\iota_n^B$. Now use the fact that there is only one morphism $\varphi: A_n \oplus B_n \rightarrow C_{n-1}$ such that $\varphi \iota_n^A = \alpha_{n-1}\partial_n^\oplus\iota_n^A$ and $\varphi \iota_n^B = \alpha_{n-1}\partial_n^\oplus\iota_n^B $.
The problem is that Hatcher does not properly define the concepts of chain maps and chain homotopies. He introduces them "en passant" when considering induced maps $f_\sharp :C_n(X) \to C_n(Y)$ and prism operators $P : C_n(X) \to C_{n+1}(Y)$ and leaves the technical details of a general definition to the reader.
It is clear that a chain map $\mathbf f : \mathbf C \to \mathbf D$ between chain complexes $\mathbf C, \mathbf D$ is a collection of group homomorphisms $f_n : C_n \to D_n$, $n \ge 0$, such that $\partial_{D,n} f_{n} = f_{n-1} \partial_{C,n}$ for all $n \ge 0$.
According to Hatcher on p. 113, a chain homotopy $\mathbf P : \mathbf C \to \mathbf D$ between chain maps $\mathbf f, \mathbf g : \mathbf C \to \mathbf D$ is a collection of group homomorphisms $P_n:C_n \to D_{n+1}$ such that $f_n - g_n = \partial_{n+1} P_n + P_{n-1}\partial_n$ for all $n$; but he is not specific about $n$. Clearly we need it for all $n \ge 0$ and this requires $P_{-1} : C_{-1} \to D_0$. Though not explicitly stated, it should be clear that Hatcher understands $C_{-1} = 0$ since this is the range of $\partial_0$. And since the zero map is the unique map $0 \to D_0$, we automatically have $P_{-1} = 0 : 0 \to D_0$.
Thus, formally, $\mathbf P$ is a collection of group homomorphisms $P_n:C_n \to D_{n+1}$, $n \ge -1$, such that $f_n - g_n = \partial_{n+1} P_n + P_{n-1}\partial_n$ for all $n \ge 0$.
A more general approach would be to define a chain complex as a collection of abelian groups $C_n$ and group homomorphisms $\partial_n : C_n \to C_{n-1}$, $n \in \mathbb Z$, such that $\partial_n \partial_{n+1} = 0$ for all $n$. It should now be obvious how to define chain maps and chain homotopies for such general chain complexes. Then we can say that the chain complexes occuring in simplicial and singular homology are "nonnegative" which means that $C_n = 0$ for $n < 0$.
Best Answer
No, that's not true. A chain homotopy equivalence is not chain map with an inverse; it is something weaker (namely it is a chain map with an "inverse up to chain homotopy," exactly the way it sounds). In particular you're confused about which direction is easy: the easy direction is that if a chain map has an inverse then it is a chain homotopy equivalence. The other direction is false.