Let $G$ be a group with $n$ elements, and let $\chi_R(g)$ be the trace of the matrix for $g$ in the representation $R$. So we have a vector for $R$: $$(\chi_R(g_1),\ldots,\chi_R(g_n))$$ which I'll denote just $\chi_R$. Now define an inner product on these vectors: for any representations $R$ and $S$, $$\langle \chi_R,\chi_S\rangle = \frac{1}{n}\sum_{g\in G}\chi_R(g)\overline{\chi_S(g)}$$ One of the big results about characters is that for any two inequivalent irreducible representations, $\langle \chi_R,\chi_S\rangle = 0$, and for any irreducible $R$, $\langle \chi_R,\chi_R\rangle = 1$. This is known as the orthogonality of irreducible characters.
Now suppose a representation $T$ is reducible. To keep notation simple, let's do the case where $T$ is the direct sum of two irreducible representations $R$ and $S$ (possibly equivalent). It's not hard to see that then $\chi_T = \chi_R+\chi_S$. By linearity of the inner product and the orthogonality just mentioned, if $R$ and $S$ are inequivalent $$\langle \chi_T,\chi_T\rangle = \langle \chi_R+\chi_S,\chi_R+\chi_S\rangle = 1+1 = 2$$ On the other hand, if $R$ and $S$ are equivalent, then $\chi_R = \chi_S$ (another big result about characters), so $\chi_T=2\chi_R$ and $$\langle \chi_T,\chi_T\rangle = 4\langle \chi_R,\chi_R\rangle = 4$$ The same argument works if $T$ is a sum of more than two irreducible characters, the notation just gets a bit messier.
So irreducible characters are characterized by $\langle \chi_R,\chi_R\rangle = 1$.
Here are a couple of examples. First, the simplest reducible case: the trivial 2-dim representation of the group with one element, $G=\{1\}$. If $V$ is the vector space, with basis (say) $\{v_1,v_2\}$, then $V$ is the direct sum $[v_1]\oplus[v_2]$ (I'm using [] for "the span of", so $V=[v_1,v_2]$). Now $[v_1]$ and $[v_2]$ are obviously invariant subspaces, so we have an invariant direct-sum decompositon, which makes the representation reducible. This reducibility shows up in the matrix for the sole element of $G$, which is $$\left[\begin{align*}\fbox{1} & 0 \\ 0 & \fbox{1}\end{align*}\right]$$ The boxes are the matrices in the 1-dim representation of $G$. Notice the block form: $$\left[\begin{align*}\fbox{$M_{11}$} & 0 \\ 0 & \fbox{$M_{22}$}\end{align*}\right]$$ Writing $T$ for the 2-dim rep, $\langle \chi_T,\chi_T\rangle = 4$. Notice that this a special case of the $\chi_T=2\chi_R$ situation described above.
Second example: let $G=S_3$. The 2-dim rep for $G$ comes from treating $G$ as the symmetry group of an equilateral triangle: each of the symmetries can be realized by an invertible linear transformation of the plane. (They also happen to be isometries.) Let's say the elements of $G$ are $\{1,r,r^2,f_1,f_2,f_3\}$ where $r$ is a $120^\circ$ rotation and $f_1,f_2,f_3$ are the three "flips" (reflexions). The vector $\chi_T$ looks like this: $(2, -1, -1, 0, 0, 0)$. (I leave it as an exercise to show that these entries are correct.) Now we compute:
$$\langle \chi_T,\chi_T\rangle = \frac{1}{6}\left(2^2+(-1)^2+(-1)^2+0^2+0^2+0^2\right) = 1$$ So the representation $T$ is irreducible, which is pretty easy to see directly.
I'm not familiar with Hill's book, but I would imagine it has the results about characters I mentioned. There is a set of on-line notes I'd recommend: Finite Groups
.
I just had this same question, and I think the answer is NO.
Let me first restate the question, since nobody seems to have understood how to phrase it.
Let $\rho: G \to C^{nxn}$ be a representation.
All equivalent representations are given by $M^{-1}\rho M$ for invertible matrices $M$. The "Weyl unitary trick" shows that for any $\rho$, there is an equivalent unitary representation. The question is: is this unique?
The answer is NO.
Suppose $\rho$ is a unitary representation.
Let U be a unitary matrix. Consider $U^* \rho U$. This is also unitary:
$$ (U^* \rho(g) U) (U^* \rho^*(g) U) = U^* \rho(g) \rho^*(g) U = U^* I U = I.$$
I hope this helps (assuming I understood the question correctly).
Best Answer
A commutative group will always contain $\vert G \vert$ 1-dimensional irreducible representations since $\sum_{i=1}^{\vert G\vert}1^2 = \vert G\vert$ this is due to every conjugacy class being unique as consequence of commutativity $a^{-1}b{a} = b$ thus the action by conjugation only yields $b$ as the unique member of the conjugacy class $b$.
This is strongly connected to character theory and a good article to start with is the associated to Dirichlet characters of multiplicative groups of integers mod $n$. https://en.wikipedia.org/wiki/Dirichlet_character#A_few_character_tables