You indicated that the problem
Let $k$ be a positive integer and let $p$ be a prime other than $2$ or $5$.
Show that the only solutions, up to congruence, of $x^2 \equiv 25\;(\text{mod}\;p^k)$ are $x \equiv \pm 5\;(\text{mod}\;p^k)$.
is the underlying problem which you are trying to solve.
For the above problem, there is no need to consider quadratic residues.
Instead, we can argue as follows . . .
Suppose $x$ is an integer such that $x^2 \equiv 25\;(\text{mod}\;p^k)$.
Note that $x+5$ and $x-5$ can't both be divisible by $p$, else their difference
$$
(x+5)-(x-5)=10
$$
would be divisible by $p$, contrary to $p\ne 2,5$.
\begin{align*}
\text{Then}\;\;&
x^2 \equiv 25\;(\text{mod}\;p^k)
\\[4pt]
\implies\;&
p^k{\,\mid\,}x^2-25
\\[4pt]
\implies\;&
p^k{\,\mid\,}(x+5)(x-5)
\\[4pt]
\implies\;&
p{\,\mid\,}(x+5)(x-5)
\\[4pt]
\implies\;&
p{\,\mid\,}(x+5)\;\text{or}\;p{\,\mid\,}(x-5)\;\text{but not both}
\\[4pt]
\end{align*}
Then since $p^k{\,\mid\,}(x+5)(x-5)$ and exactly one of $x+5,x-5$ is divisible by $p$, it follows that exactly one of $x+5,x-5$ is divisible by $p^k$.
Therefore $x \equiv \pm 5\;(\text{mod}\;p^k)$, as was to be shown.
Best Answer
If such an element exists it would have order $4$, so by Lagrange's theorem $4|p-1$ and thus $p\equiv 1\mod 4$.
If $p\equiv 1\mod 4$ we can use Wilson's theorem to explicitly write down an element that squares to $-1$: $$ -1\equiv(p-1)!\equiv 1\cdot\ldots\cdot \frac{p-1}{2}\frac{p+1}{2}\cdot\ldots\cdot(p-1)\equiv\left(\left(\frac{p-1}{2}\right)!\right)^2\cdot(-1)^{\frac{p-1}{2}}\equiv\left(\left(\frac{p-1}{2}\right)!\right)^2 $$