Just to sum up, mostly for my own reference, but I thought others might find it useful. (I am new to the site, so please excuse me if this shouldn't be an answer...)
First some preliminary notions:
For a topological space $X$, an $n$-simplex in $X$ is a continuous map $\Delta^n \to X$ from the standard geometric $n$-simplex $\Delta^n$ into $X$. The maps $d^i: \Delta^{n-1} \to \Delta^{n}$, sends $\Delta^{n-1}$ to the face of $\Delta^n$ sitting opposite the $i$th vertex of $\Delta^n$.
An ordered $n$-simplex is a partially ordered set $n_+ = \{ 0 < 1 < \cdots < n \}$. The $n+1$ elements of $n_+$ is called the vertices of $\sigma$. The subsets of $n_+$ are called the faces of $\sigma$. There are morphisms of simplices $d^i: (n-1)_+ \to n_+$ called coface maps, given by $d^i((n-1)_+) = \{ 0 < 1 < \dots < î < \cdots < n \}$ omitting the $i$th vertex of $n_+$.
Then for the two homologies:
The singular (unreduced) chain complex on a space $X$, is the chain complex
$$\cdots \xrightarrow{\partial_{n+1}} C_n(X) \xrightarrow{\partial_n} C_{n-1}(X) \xrightarrow{\partial_{n-1}} \cdots C_1(X) \xrightarrow{\partial_1} C_0(X) \to 0$$
where $C_n(X)$ is the free abelian group $\mathbb{Z}[S_n(X)]$ generated by the set $S_n(X) = \{ \sigma : \Delta^n \to X \}$ of all $n$-simplices in $X$ (i.e. the set of all continuous maps $\Delta^n \to X$). The boundary maps $\partial_n : C_n(X) \to C_{n-1}(X)$ is given by $\partial_n (\sigma) = \sum_{i=0}^{n}(-1)^i \sigma d^i : \Delta^{n-1} \to \Delta^n \to X$.
The $n$th homology group $H_n(X) = \ker(\partial_n) / \text{im}(\partial_{n+1})$ of this complex is the $n$th singular homology group of $X$.
A simplicial complex $S$ is a set $S = \bigcup_{n=0}^{\infty} S_n$ where $S_n = S(n_+)$ being a set of ordered $n$-simplices, such that a face of any simplex in $S$ is itself a simplex in $S$. The simplicial chain complex
$$\cdots \xrightarrow{\partial_{n+1}} C_n(S) \xrightarrow{\partial_n} C_{n-1}(S) \xrightarrow{\partial_{n-1}} \cdots C_1(S) \xrightarrow{\partial_1} C_0(S) \to 0$$
consists of the free abelian groups $C_n(S) = \mathbb{Z}[S_n]$ generated by the $n$-simplices. The boundary map $\partial_n : C_n(S) \to C_{n-1}(S)$ is given by $\partial_n(\sigma) = \sum_{i=0}^n (-1)^i d_i \sigma$ where $d_i = S(d^i) : S_n \to S_{n-1}$ is the face maps $d_i(\sigma) = \sigma \circ d^i$.
The $n$th homology groups of this complex $H^\Delta_n(S) = \ker(\partial_n) / \text{im}(\partial_{n+1})$ is the $n$th simplicial homology group of $S$.
Lastly we have the realization of $S$, $|S| = \coprod (S_n \times \Delta^n) / \left((d_i \sigma, y) \sim (\sigma, d^iy) \right)$ for all $(\sigma, y) \in S_n \times \Delta^{n-1}$, where $d_i \sigma \times \Delta^{n-1}$ is identified with the $i$'th face of $\sigma \times \Delta^n$.
Then if you want to say something about a specific space $X$, you need to find a simplicial complex $S$, whose realization is homeomorphic to $X$ (i.e. you triangulate $X$ and find the homology groups of the resulting simplicial complex).
NOTE: Feel free to edit any mistakes and clarify where you find it necessary. I'm still not 100% comfortable with it yet..
This is just an elaboration on PVALs comment.
Given a short exact sequence $0 \to A \to B \to C \to 0$ of chain complexes, there is an induced long exact sequence in homology. This is constructed out what is known as the snake lemma: https://en.wikipedia.org/wiki/Snake_lemma
If you look in the construction of the boundary map, you will see that a key step is to use the differential of $B$. This can be interpreted topologically.
First lets just recall the construction of the relative chain complex short exact sequence. The idea is that the inclusion $A \to X$ induces an inclusion of (singular) chain complexes, and one can formally take the quotient to get the relative chain complex. (The quotient is also free, since a simplex is killed in the quotient iff it lied entirely in $A$, so we create no torsion after this quotient.)
Now we start with some representative $\alpha$ of an element in the relative homology group $H_n(X,A)$, which is some chain in $X$ whose boundary lies in $A$. $\partial \alpha$ is some chain lying in $A$, and since it was a boundary of a chain in $X$, we know that it is a cycle. But it may not be a boundary of some chain lying in $A$, hence could be some interesting homology class in $H_{n-1}(A)$.
It is useful to imagine the disc $D = X$ with boundary $S^1 = A$. The disc is a relative homology class in $H_2(X,A)$. It's boundary is a homology class in $H_1(A)$.
The long exact sequence asserts that in general a complete set of representatives for the homology classes in the kernel of the map $H_{n-1}(A) \to H_{n-1}(X)$ are obtained this way. But this really makes sense, since some $\beta$ being in the kernel of that map means that there appeared some chain $\alpha$ in $X$ filling in the holes of $\beta$, i.e. it became a boundary, or symbolically $\partial(\alpha) = \beta$. But now $\alpha$ is a chain in $C_n(X)$ whose boundary lies in $A$, hence it represents some relative cycle in $H_n(X,A)$, and its image under the map described before is exactly $\beta$.
Hope that is helpful.
Best Answer
$C_n(X)$ is the set of all formal sums of $n-$simplices in $X$, an element of $C_n(X)$ is given by a finite sum $\sum n_i \phi_i$ for some $\phi_i : \Delta^n \rightarrow X$ and $n_i \in \mathbb Z$. This is just a formal sum. And yes he means that $C_n(X)$ is the free abelian group generated by the set of $\textbf{all}$ n-simplices.
And that definition of the boundary map is really, really misleading, I could 100% see how you would think that deleting different $i$ give different domains. And according to that definition that is actually true. A better definition is $\partial \sigma = \sum (-1)^i \sigma \circ d_i$ where $\sigma : \Delta^n \rightarrow X$ and
$$d_i: \Delta^{n-1} \rightarrow \Delta^n : (x_0,...,x_{n-1}) \mapsto (x_0,...,x_{i-1}, 0,x_{i},...,x_{n-1})$$
then the resulting maps have the same domain and the resulting formal sum is in $C_{n-1}(X)$.
The map $d_i$ is the same as the map that @bart described in his comment by the way.