(posting an answer since my comment was too long)
Part a)
I'm afraid you're going a bit too quick between the proof that $|G|=p ^n$ and the element all have same order and then the conclusion that $G$ must be an elementary abelian group.
For instance, the Heisenberg group (upper triangular 3x3 matrices with ones on the diagonal) over $\mathbb{F}_p$ has order $p^3$ and all elements have order $p$ for $p \geq 3$.
However, you can still use the hypotheses to reach this conclusion. Indeed, you have proved that $G$ is a $p$-group. It is well known that $p$-group have non trivial center. So there is a central non trivial element and all non trivial elements are conjugate to it. All conjugates of a central element are central and identity is central also. The group is thus equal to its center.
You can then conclude that $G$ is abelian and then elementary because all non-trivial elements have order $p$.
Part b)
An elementary group of order $p^n$ is a vector space over the finite field $\mathbb{F}_p$. If you consider a automorphism, it translates into a vector space homomorphism, which must be injective. This confirms that $Aut(G)=GL_n(\mathbb{F}_p)$.
Determining when $GL_n(\mathbb{F}_p)$ is solvable is a more complex task. I will not provide a precise proof but rather present the ideas and references to reach such a proof.
Let us start with the determinant morphism. We have a morphism $GL_n(\mathbb{F}_p) \longrightarrow \mathbb{F}_p^*$. The multiplicative group of a finite field is a cyclic group of order $p-1$ (see Wikipedia page on finite fields) and thus is solvable. The kernel of the determinant map (morphism) is denoted $SL_n(\mathbb{F}_p)$. We thus have $Aut(G)$ is solvable iff $SL_n(\mathbb{F}_p)$ is.
Now, consider the subgroup of scalar matrices (all entries zero except on diagonal) with determinant one. This group is also solvable (as a subgroup of the group of all diagonal matrices, which is isomorphic to $C_{p-1}^n$). So we can conclude that $Aut(G)$ is solvable iff the quotient, noted $PSL_n(\mathbb{F}_p)$ is.
Last, it is a classical complex but well-known result that $PSL_n(\mathbb{F}_p)$ is simple non abelian except when $n=1$ and also when $n=2$ and $p \in \{2,3\}$. A reference, among many others, is Robert A. Wilson / The Finite Simple Groups / Graduate Texts in Mathematics / Springer.
To conclude, $Aut(G)$ is solvable only when $n$ is 1for all values of $p$ and when $n$ is 2 and $p$ is 2 or 3.
Best Answer
Because the action of $\mathrm{Aut}(G)$ on $G-\{1\}$ is transitive, we know that every nontrivial element has the same order. In particular, $G$ must be a group of exponent $p$ for some prime $p$.
Since $G$ is a $p$-group, it has nontrivial center. If $g\in Z(G)-\{1\}$, then every image of $g$ under an automorphism is nontrivial and central, so transitivity of the action of $\mathrm{Aut}(G)$ gives us that every nontrivial element is central, so $G$ is an abelian $p$-group of exponent $p$. Hence $G$ is an elementary abelian $p$-group.
Thus, $G\cong C_p^n$ for some $n\geq 0$.
If $n\geq 2$, then the automorphism that swaps the first and second coordinate and leaves the rest fixed has a nontrivial fixed point (the element $(x,x,1,\ldots,1)$), so the action is not free.
Thus, if the action is transitive and free, then $n\leq 1$.
Conversely, for $n=0$, the action on $G-\{1\}$ is vacuously regular. For $n=1$, either $\mathrm{Aut}(G)$ is trivial (when $p=2$), or else an automorphism sends the generator $x$ to $x^a$ for some $a$, $1\leq a\lt p$. If $x^r = x^{ar}$, then $r\equiv ar\pmod{p}$, hence $1\equiv a\pmod{p}$, so the automorphism is the identity element of $\mathrm{Aut}(G)$, showing the action is free. The action is transitive on $G-\{1\}$ since $x\mapsto x^a$ yields an automorphism for each $a$, $1\leq a\lt p$.