From Silverman's A Friendly Introduction to Number Theory, exercise 12.3 (This is not homework). We start with a prime number $p$ and let
$$\sum_{n=1}^{p-1}{\frac{1}{n}} = \frac{A_p}{B_p}$$
where $\frac{A_p}{B_p}$ is a reduced form fraction. The exercise asks to find a simple way to express $A_p$ (mod $p^2$).
I understand that $A_p$ is $0$ (mod $p$), which can be proved by rewriting $A_p$
$$A_p = \sum_{n=1}^{p-1}{\frac{(p-1)!}{n}}$$
and noting that each term in the sum is in the form
$$\frac{(p-1)!}{n}$$
which allows us to write
$$
(p-1)! \equiv -1 \text{ (mod } p)\\
\frac{(p-1)!}{n} \cdot n \equiv -1 \text{ (mod } p)\\
\frac{(p-1)!}{n} \equiv -n^{-1} \text{ (mod } p).
$$
Since every number between $1$ and $p$ – $1$ is represented by $-n^{-1}$ (mod $p$) for some unique $n$ between $1$ and $p$ – $1$, we can write
$$A_p \equiv T_{p-1} = \frac{(p-1)p}{n} \equiv 0 \text{ (mod } p).$$
However, the problem only asks for $A_p$ (mod $p^2$), for which I don't know where to proceed. I've tested every prime number up to $p = 37$ and found that with the exception of $p = 3$ every value of $A_p$ is $0$ (mod $p^2$).
If someone who understands what I'm missing can point me in the right direction or give me a hint I'd appreciate it.
EDIT
It's been suggested that Wolstenholme's theorem can be used to finish the proof. The connection between the problem here and Wolstenholme's theorem is interesting so I will try to study Wolstenholme's theorem. However, it doesn't seem like a likely path to have taken from where I'm at in my understanding. I'm also skeptical Silverman had intended for the audience of his book to discover Wolstenholme's theorem as a lemma to proving that $A_p$ is $0$ (mod $p^2$).
Best Answer
I eventually found a solution. The trick is to first factor out a $p$, then to show that the remaining expression is still $0$ (mod $p$).
\begin{align*} \sum_{n=1}^{p-1}{\frac{1}{n}} &= \sum_{n=1}^{\frac{p-1}{2}}{(\frac{1}{n} + \frac{1}{p-n})} \\ &= \sum_{n=1}^{\frac{p-1}{2}}{\frac{p}{n(p-n)}} \end{align*}
After removing $p$ we obtain
\begin{align*} \sum_{n=1}^{\frac{p-1}{2}}{\frac{1}{n(p - n)}} &\equiv \sum_{n=1}^{\frac{p-1}{2}}{\frac{1}{n(0 - n)}} &(\text{mod } p) \\ &= \sum_{n=1}^{\frac{p-1}{2}}{-n^{-2}} \\ &\equiv -2\sum_{n=1}^{p-1}{n^{-2}} &(\text{mod } p) \end{align*}
Since $\sum_{n=1}^{p-1}{n^{-2}}$ is just a reordering of the terms of $\sum_{n=1}^{p-1}{n^{2}}$ (mod $p$) we can write
\begin{align*} -2\sum_{n=1}^{p-1}{n^{-2}} &\equiv -2\sum_{n=1}^{p-1}{n^{2}} &(\text{mod } p) \\ &= -2\frac{(p-1)p(2(p-1) + 1)}{6} \\ &= -\frac{(p-1)p(2p - 1)}{3} \\ &\equiv 0 & (\text{mod } p) \text{ if } p \ne 3 \end{align*}