$\def\ZZ{\mathbb{Z}}$I now understand the issue that user10676 is concerned about: For $X$ a reasonable topological space, why is $H^k(X, \ZZ_{\ell})$ isomorphic to $\lim_{\infty \leftarrow n} H^k(X, \ZZ/\ell^n)$? I thought that this would be most easily done from the universal coefficient theorem but, now that I try to write out a proof, I find it easiest to do directly.
Every complex algebraic variety is homeomorphic to a finite simplicial complex. (If I recall correctly, you can find a proof in "Algorithms in Real Algebraic Geometry".) So I'll use simplicial rather that singular cohomology. We work with a fixed triangulation for the rest of this proof.
Let $C^k$ be the group of $k$-co-chains with $\ZZ$ coefficients. Let $Z^k$ be the co-cycles and $B^k$ the co-boundaries. So $H^k(X, \ZZ) = Z^k/B^k$. Let the finitely generated abelian group $H^k(X, \ZZ)$ be $\bigoplus_{1 \leq i \leq r} \ZZ/d_i \oplus \ZZ^{\oplus s}$ where $d_1$, $d_2$, ..., $d_r$ is a sequence of positive integers where $d_1$ divides $d_2$ divides ... divides $d_r$. The following lemma is useful in many computations about cohomology:
Lemma Let $C^{k-1} \to C^{k} \to C^{k+1}$ be a complex of free finitely generated abelian groups. Then we can write this complex as a direct sum of complexes of the following forms:
$$\ZZ \longrightarrow 0 \longrightarrow 0 \quad (1)$$
$$0 \longrightarrow \ZZ \longrightarrow 0 \quad (2)$$
$$0 \longrightarrow 0 \longrightarrow \ZZ \quad (3)$$
$$\ZZ \stackrel{d}{\longrightarrow} \ZZ \longrightarrow 0 \quad (4)$$
$$0 \longrightarrow \ZZ \stackrel{d}{\longrightarrow} \ZZ \quad (5)$$
where $d$ is a positive integer. (The case $d=1$ is permitted.)
Proof First, choose arbitrary bases for $C^{k-1}$ and $Z^k$. The map $C^{k-1} \to Z^k$ is given by an integer matrix. Now change bases so that this matrix is in Smith normal form. In these bases, $C^{k-1} \to Z^k \to 0$ is a sum of complexes of types (1), (2) and (4).
Now, $B^{k+1}$ is a subgroup of the free $\ZZ$-module $C^{k+1}$, so it is free. Thus, we can choose a splitting of $C^k \to C^k/Z^k \cong B^{k+1}$. Let $A$ be the image of this splitting, so $C^k \cong Z^k \oplus A$. Put the map $A \to C^{k+1}$ into Smith normal form as before. Then $0 \to A \to C^{k+1}$ is a direct sum of complexes of the form (3) and (5). (Since $A \to C^{k+1}$ is injective, there are no zeroes on the diagonal of the Smith normal form, and (2) does not occur.) Our original complex is the direct sum of $C^{k-1} \to Z^k \to 0$ and $0 \to A \to C^{k+1}$, so it breaks up as a direct sum of complexes of the five types listed above. $\square$
If we now change coefficients to a new abelian group $G$, we get the same complexes with $\ZZ$ replaced by $G$. All our computations distribute over direct sum, so we just need to work out what each of these five complexes do for the groups $G = \ZZ_{\ell}$ and $G = \ZZ/\ell^n$.
For complexes (1) and (3), we get $0$ in both cases.
For complex (2), we get $\lim_{\infty \leftarrow n} \ZZ/\ell^n$ on one side and $\ZZ_{\ell}$ on the other (with the obvious surjections in the inverse limit).
For complex (4), let $d = \ell^i m$ with $GCD(\ell, m) =1$. For $n \geq i$, we get $\lim_{\infty \leftarrow n} \ZZ/\ell^i$ on one side and $\ZZ/\ell^i$ on the other, where the maps in the inverse limit are the identity for $n > i$.
Complex (5) is the hard one. On the $\ZZ_{\ell}$ side, we get $0$. Write $d = \ell^i m$ as above. For $n \geq i$, we again get $\lim_{\infty \leftarrow n} \ZZ/\ell^i$. However, this time the map in the inverse limit is multiplication by $\ell$ (for $n>i$). This inverse limit is $0$, so we win.
Remark It is probably worth stating the general version of this result: If $R$ is a PID and $C^{\bullet}$ is a complex of free $R$-modules, then $C^{\bullet}$ can be written as a direct sum of complexes of the forms $\cdots \to 0 \to 0 \to R \to 0 \to 0 \to \cdots$
and $\cdots \to 0 \to 0 \to R \stackrel{d}{\longrightarrow} R \to 0 \to \cdots$ for various $d \in r$, and where the nonzero terms can occur in any position. The proof is basically the same: submodules of a free module over a PID are free; surjections to free modules split; Smith normal form is valid over a PID. A lot of modern ring theory can be thought of as classifying the types of complexes which exist over different rings.
Best Answer
Here is a pedestrian way to see it:
Firstly, note that the homologies $H_{*}(X, \mathbb{Z}_{\ell})$ are finitely generated: this is a consequence of the fact that quasi-projective complex varieties can be modelled by finite CW complexes. This means, since $\mathbb{Z}_{\ell}$ is a pid, that we may decompose $H_{i}(X, \mathbb{Z}_{\ell})=F_i \oplus T_i$ where $F_i$ is a finite free $\mathbb{Z}_{\ell}$-module and $T_i$ is torsion, i.e. $\ell^NT_i=0$ for some $N$.
Now, there is a universal coeff. theorem for cohomology, which says that for a pid $R$ and an $R$-module $A$, one has for every $i$ the short exact sequence $$0 \rightarrow \mathrm{Ext}^1_R(H_{i-1}(X, R),A)\rightarrow H^i(X, A)\rightarrow \mathrm{Hom}_R(H_{i}(X, R),A) \rightarrow 0.$$
We can use this fact twice: first for $A=\mathbb{Q}_{\ell}$. In that case, since $\mathbb{Q}_{\ell}$ is divisible hence injective, $\mathrm{Ext}^1_{\mathbb{Z}_{\ell}}(H_{i-1}(X, \mathbb{Z}_{\ell}),\mathbb{Q}_{\ell})=0$ and we get an isomorphism $H^i(X, \mathbb{Q}_{\ell})\simeq \mathrm{Hom}_{\mathbb{Z}_{\ell}}(H_{i}(X, \mathbb{Z}_{\ell}), \mathbb{Q}_{\ell})$.
Secondly, we can use it for $A=\mathbb{Z}_{\ell}$ itself. Now the first term $\mathrm{Ext}^1_{\mathbb{Z}_{\ell}}(H_{i-1}(X, \mathbb{Z}_{\ell}),\mathbb{Z}_{\ell})$ is not zero in general, but, it is equal to $\mathrm{Ext}^1_{\mathbb{Z}_{\ell}}(F_{i-1},\mathbb{Z}_{\ell})\oplus \mathrm{Ext}^1_{\mathbb{Z}_{\ell}}(T_{i-1},\mathbb{Z}_{\ell}),$where $\mathrm{Ext}^1_{\mathbb{Z}_{\ell}}(F_{i-1},\mathbb{Z}_{\ell})=0$ because $F_{i-1}$ is free and $\mathrm{Ext}^1_{\mathbb{Z}_{\ell}}(T_{i-1},\mathbb{Z}_{\ell})$ is torsion because $T_{i-1}$ is. So in this case, the map $H^i(X, \mathbb{Z}_{\ell})\rightarrow \mathrm{Hom}_{\mathbb{Z}_{\ell}}(H_{i}(X, \mathbb{Z}_{\ell}), \mathbb{Z}_{\ell})$ is only surjective with a trosion kernel, which, however, implies that $H^i(X, \mathbb{Z}_{\ell})\otimes_{\mathbb{Z}_{\ell}}\mathbb{Q}_{\ell}\simeq \mathrm{Hom}_{\mathbb{Z}_{\ell}}(H_{i}(X, \mathbb{Z}_{\ell}), \mathbb{Z}_{\ell})\otimes_{\mathbb{Z}_{\ell}}\mathbb{Q}_{\ell}$.
So now the last thing that remains is to show that $\mathrm{Hom}_{\mathbb{Z}_{\ell}}(H_{i}(X, \mathbb{Z}_{\ell}), \mathbb{Z}_{\ell})\otimes_{\mathbb{Z}_{\ell}}\mathbb{Q}_{\ell} \simeq \mathrm{Hom}_{\mathbb{Z}_{\ell}}(H_{i}(X, \mathbb{Z}_{\ell}), \mathbb{Q}_{\ell})$. Dissecting $H_{i}(X, \mathbb{Z}_{\ell})=F_i\oplus T_i$ again, similarly the torsion part contribution on both sides vanishes since the targets in the Hom groups are torsion free, and so we are left with showing that $\mathrm{Hom}_{\mathbb{Z}_{\ell}}(F_i, \mathbb{Z}_{\ell})\otimes_{\mathbb{Z}_{\ell}}\mathbb{Q}_{\ell} \simeq \mathrm{Hom}_{\mathbb{Z}_{\ell}}(F_i, \mathbb{Q}_{\ell})$. But this is now easy, $F_i \simeq \mathbb{Z}_\ell^{n}$ is finite free and thus both sides are easily seen to be isomorphic to $\mathbb{Q}_{\ell}^n$.