First, some notation. Let us represent an element of $\mathbb{Z}/p^n\mathbb{Z}$ by the pair $(a,n)$, where $a\in\mathbb{Z}$; so $(a,n)=(b,n)$ if and only if $a\equiv b\pmod{p^n}$.
We write $\mathbb{Z}_{p^{\infty}}$ for the direct limit; the elements of the direct limit are equivalence classes of the form $[a,n]$, $a\in\mathbb{Z}$, $n\geq 1$, with $[a,n]=[b,m]$ if and only if (i) $(p^{m-n}a,m)=(b,m)$ if $m\geq n$; or (ii) $(a,n) = (p^{n-m}b,n)$ if $m\lt n$. Addition is defined by
$$[a,n]+[b,m] = \left\{\begin{array}{ll}
\ [a+p^{n-m}b,n] &\text{if }n\geq m;\\
\ [p^{m-n}a+b,m] &\text{if }n\lt m.
\end{array}\right.$$
The universal property of the direct limit means that if $G$ is any group, and we have group homomorphisms $f_n\colon \mathbb{Z}/p^n\mathbb{Z}\to G$ such that for all $n\leq m$, $f_n(a,n) = f_m(p^{m-n}a,m)$, then there exists a unique $f\colon\mathbb{Z}_{p^{\infty}}\to G$ such that $f[a,n] = f_n(a)$.
First, we need to show that $\mathbb{Z}_{p^{\infty}}$ satisfies the conditions given; then we want to show that given any group $G$ that satisfies the given conditions, we have an isomorphism from $G$ to $\mathbb{Z}_{p^{\infty}}$ (alternatively, show that $G$ has the universal property associated to the direct limit).
Property 1. The unique infinite $p$-group in which every element has exactly $p$ distinct $p$th roots.
Note that $\mathbb{Z}_{p^{\infty}}$ is infinite (it contains at least $p^n$ elements for each $n$. Also, each element has order a power of $p$, since $p^n[a,n] = [p^na,n] = [0,n]$ (since $p^na\equiv 0\pmod{p^n}$). So $\mathbb{Z}_{p^{\infty}}$ is an infinite $p$-group.
Also, every element of $\mathbb{Z}_{p^{\infty}}$ has at least $p$ distinct $p$th roots: let $[a,n]\in\mathbb{Z}_{p^{\infty}}$. Then for each $k$, $0\leq k\lt p$, we have
$$p[a+kp^n,n+1] = [pa+kp^{n+1},n+1] = [pa,n+1] = [a,n];$$
and $[a+kp^n,n+1] = [a+\ell p^n,n+1]$ if and only if $kp^n\equiv \ell p^n\pmod{p^{n+1}}$, if and only if $k\equiv \ell\pmod{p^n}$. Since we are assuming $0\leq k,\ell\lt p$, we conclude that $k=\ell$, so $[a,n]$ has at least $p$ distinct $p$th roots.
To show there are exactly $p$ distinct $p$th roots, first we show there are exactly $p$ distinct elements of order dividing $p$. If $p[b,m] = [0,m]$, then $pb\equiv 0\pmod{p^m}$, hence $p^{m-1}|b$. Writing $b=p^{m-1}k$ we have $[b,m] = [kp^{m-1},m] = [k,1]$. Thus, every element of order $p$ must be one of $[k,1]$ with $0\leq k\lt p$, so there are at most $p$ elements of order dividing $p$; each of these is distinct, so there are exactly $p$ elements of order dividing $p$.
Now suppose that $[b,m]$ is a $p$th root of $[a,m]$. Then for each $k$, $0\leq k\lt p$, we have
$$p\Bigl( [b,m] - [a+kp^n,n+1]\Bigr) = 0.$$
Therefore,
$$\Bigl\{ [b,m] - [a+kp^n,n+1]\Bigm| k=0,1,\ldots,p-1\Bigr\} = \Bigl\{ [0,1], [1,1],\ldots,[p-1,1]\Bigr\}$$
(since the $[b,m]-[a+kp^n,n+1]$ are pairwise distinct and all elements of order $p$). Therefore, there exists $k$ such that $[b,m]-[a+kp^n,n+1] = [0,1]$, hence $[b,m]=[a+kp^n,n+1]$, proving that the aforementioned $p$th roots of $[a,n]$ are the only $p$th roots of $[a,n]$. Thus, $[a,n]$ has exactly $p$ distinct $p$th roots. This proves that $\mathbb{Z}_{p^{\infty}}$ is an infinite $p$-group in which every element has exactly $p$ distinct $p$th roots.
The next step is to show that any infinite $p$-group in which every element has exactly $p$ distinct $p$th roots is isomorphic to the direct limit. Let $G$ be such a group.
First, we construct maps $f_m\colon \mathbb{Z}/p^m\mathbb{Z}\to G$ for each $m$ inductively, ensuring at each step that the maps form a consistent system. Since we do not know ahead of time whether $G$ is abelian, I will use multiplicative notation for $G$. For each $n\in\mathbb{N}$, let
$$G[p^n] = \{g\in G\mid g^{p^n}=1\}.$$
Note that $G[p]\subseteq G[p^2]\subseteq G[p^3]\subseteq\cdots$.
Consider $G[p] = \{g\in G\mid g^p = 1\}$. We know that $G[p]$ contains $p$ distinct elements (since by assumption, $1$ has exactly $p$ distinct $p$th roots in $G$). Let $x_1\in G[p]$, $x_1\neq 1$. Then $\langle x_1\rangle$ is cyclic of order $p$, and every element is of exponent $p$. Therefore, $G[p]=\langle x_1\rangle$. Define $f_1\colon \mathbb{Z}/p\mathbb{Z}\to \langle x_1\rangle$ by $f_1(1,1) = x_1$.
Assume that we have shown that $G[p^k]$ is a cyclic group of order $p^k$, generated by $x_k$ for $1\leq k\leq n$, and have defined maps $f_k\colon\mathbb{Z}/p^k\mathbb{Z} \to G[p^k]$ in such a way that the maps commute with the natural embeddings $\mathbb{Z}/p^k\mathbb{Z}\hookrightarrow \mathbb{Z}/p^{k+1}\mathbb{Z}$ for all $k$, $0\leq k\lt n$. In particular, we have that $x_{k+1}^p = x_k$ for each $k$.
Consider $G[p^{n+1}]$. Note that $g\in G[p^{n+1}]$ if and only if $g^p\in G[p^n]$. Since $G[p^n]$ has $p^n$ elements, and each element has exactly $p$ distinct $p$th roots, we conclude that $G[p^{n+1}]$ has $p\times p^n = p^{n+1}$ elements. Let $x_{n+1}$ be a $p$-th root of $x_n$. Then $x_{n+1}$ has order $p^{n+1}$ (since $x_n$ has order $p^n$), and since $\langle x_{n+1}\rangle \subseteq G[p^{n+1}]$, and both have $p^{n+1}$ elements, then $G[p^{n+1}]$ is a cyclic group, generated by $x_{n+1}$. Define $f_{n+1}\colon\mathbb{Z}/p^{n+1}\mathbb{Z}\to G[p^{n+1}]$ by mapping $[1,n+1]$ to $x_{n+1}$. This map is compatible with the embedding $\mathbb{Z}/p^n\mathbb{Z}\hookrightarrow \mathbb{Z}/p^n\mathbb{Z}$, and hence with all the previous embeddings as well. This completes the induction step.
Note that since $G$ is a $p$-group, if $g\in G$ then there exists $k$ such that $g\in G[p^k]$; thus, $G = \cup G[p^k]$.
Thus, we have a family of consistent maps $f_m\colon\mathbb{Z}/p^m\mathbb{Z}\to G$.
To show that $G$ has the universal property of the direct limit, let $H$ be any group, and let $h_m\colon\mathbb{Z}/p^m\mathbb{Z}\to H$ be a family of group homomorphisms which commute with the embeddings $\mathbb{Z}/p^k\mathbb{Z}\hookrightarrow \mathbb{Z}/p^{k+1}\mathbb{Z}$.
We define $h\colon G\to H$. Let $g\in G$; let $k$ be an integer such that $g\in G[p^k] = \langle x_k\rangle$. Write $g = x_k^r$, and define $h(g) = h_k(r,k)$.
Note that $h$ is well-defined: if $g = x_k^r = x_m^s$, with $k\leq m$, then we know that $x_k = x_m^{p^{m-k}}$, hence $x_k^r = x_m^{rp^{m-k}}$. Hence $s\equiv rp^{m-k}\pmod{p^m}$ (since $x_m$ is of order $p^m$). Therefore,
$$h_k(r,k) = h_m(p^{m-k}r,m) = h_m(s,m)$$
which shows $h(g)$ is well defined (we have used that the $h_n$ are a consistent system, so that $h_k(r,k) = h_{k+1}(pr,k+1)$, etc).
Also, $h$ is a group homomorphism: if $g,g'\in G$, let $k$ be such that $g,g'\in G[p^k]$. Then $gg'\in G[p^k]$ (since $G[p^k]$ is a subgroup), and so
$$h(gg') = h_k(gg') = h_k(g)h_k(g') = h(g)h(g').$$
And for every $m$, we have $h_m(a,m) = h(f_m(a,m))$ by definition, so $h$ fits into the required commutative diagrams.
Now assume that $\psi\colon G\to H$ is any group homomorphism such that $h_m = \psi\circ f_m$ for each $m$. Then $\psi(x_k) = \psi(f_k(1,k)) = h_k(1,k) = h(x_k)$. Since $G$ is generated by the $x_k$, this proves that $\psi=h$.
Thus, $G$ has the properties of the direct limit of the system
$$\mathbb{Z}/p\mathbb{Z} \to \mathbb{Z}/p^2\mathbb{Z}\to\mathbb{Z}/p^3\mathbb{Z}\to\cdots$$
and therefore there exists a (unique) isomorphism $\mathbb{Z}_{p^{\infty}}\cong G$. Thus, $\mathbb{Z}_{p^{\infty}}$ is the unique (up to unique isomorphism) infinite $p$-group in which every element has exactly $p$ distinct $p$th roots, as claimed. QED
Property 2. The unique infinite $p$ group that is locally cyclic.
We already know that $\mathbb{Z}_{p^{\infty}}$ is an infinite $p$-group. To show it is locally cyclic, it suffices to show that any two elements are contained in a cyclic subgroup (since then the subgroup they generate is itself cyclic).
Let $[a,m]$ and $[b,n]$ be elements of $\mathbb{Z}_{p^{\infty}}$, with $m\leq n$. Then $[a,m] = [p^{n-m}a,n] = p^{n-m}a[1,n]$, and $[b,n]=b[1,n]$. Hence,
$$[a,m],[b,n] \in\langle [1,n]\rangle.$$
Thus, $\mathbb{Z}_{p^{\infty}}$ is locally cyclic.
Now let $G$ be a locally cyclic infinite $p$-group. We will show that every element of $G$ has exactly $p$ distinct $p$th roots, which will establish the desired isomorphism using part 1.
Let $a\in G$, and let $|\langle a\rangle|=p^n$. Since $a$ is of finite order and $G$ is infinite, there exists $b\in G$, $b\notin \langle a\rangle$. In particular, the subgroup generated by $a$ and $b$ is cyclic, $\langle a,b\rangle = \langle c\rangle$, and $|\langle c\rangle| \gt p^n$. Thus, $\langle c\rangle\cong \mathbb{Z}/p^{n+k}\mathbb{Z}$ for some $k\gt 0$; in this subgoup $a$ has exactly $p$ distinct $p$th roots. Thus, every element of $G$ has at least $p$ distinct $p$th roots.
Now suppose that $r_1,\ldots,r_p$ are $p$ distinct $p$th roots of $a$, and let $s$ be any $p$th root of $a$. Then $\langle r_1,\ldots,r_p,s\rangle$ is cyclic of order a power of $p$, and contains $a$. Since in a cyclic group of order $p^n$ each element has at most $p$ distinct $p$th roots, then $s$ must be one of $r_1,\ldots,r_p$, proving that $a$ has exactly $p$ distinct $p$th roots.
Thus, each element of $G$ has exacty $p$ distinct $p$th roots, and so is isomorphic to $\mathbb{Z}_{p^{\infty}}$ by the first part. QED
Best Answer
How one procedes to prove these probably depends a bit on which description of the $p$-Prüfer group one is most comfortable with. So my particular arguments may not seem very natural to you, in which case this will probably not help much.
Because the subgroups are ordered by inclusion, given any $x,y\in G$, either $x\in\langle y\rangle$ or $y\in\langle x\rangle$. Take an element $x_1\in G$. The subgroup it generates cannot be all of $G$ (it is finite, since $G$ is a $p$-group), so there exists $x_2\in G$ with $x_2\notin\langle x_1\rangle$. Therefore, $x_1\in\langle x_2\rangle$. But $x_2$ cannot generate all of $G$. So there exists $x_3\in G$, $x_3\notin \langle x_2\rangle$.
Continuing this way, you obtain a (countably infinite) collection of elements of $G$, $x_1,x_2,\ldots,$ such that $x_n\in\langle x_{n+1}\rangle$, $x_{n+1}\notin\langle x_n\rangle$. By introducing suitable elements into the sequence, we may assume that $[\langle x_{n+1}\rangle\colon\langle x_n\rangle]=p$.
Now prove that the subgroup generated by all the $x_i$ is isomorphic to the $p$-Prüfer group, and that it is equal to $G$.
Let $x_1\in G$. It does not generate all of $G$, so let $y\in G$ that is not in $\langle x_1\rangle$. Then $\langle x_1,y\rangle$ is cyclic, strictly larger than $\langle x_1\rangle$; let $x_2$ be a generator of this cyclic group. Since $x_2$ does not generate all of $G$, there exists $y\in G$ not in $\langle x_2\rangle$; but $\langle x_2,y\rangle$ is cyclic, so let $x_3$ be a generator. Lather, rinse, repeat, to get an infinite sequence as above, adding elements if necessary to get each step to increase the size of the cyclic group by $p$; that is, $x_n$ is of order $p^n$.
Again, prove the group generated by the $x_i$ is the $p$-Prüfer group and equal to all of $G$.
I'd written suitable arguments for 3 and 4, but Jack has posted them as well and they are essentially the same, so I'll defer to him on them.