Claim:$G$ is cyclic.
Proof: If not, then there must be two distinct elements $a$ and $b$ that generate two distinct cyclic subgroups. Thus, $G$ must be cyclic.
From here, we know all cyclic groups are isomorphic to $\mathbb{Z}_n$, where $n = |G|$. At this point, we are done: For all cyclic groups $H$, there will exist exactly $1$ cyclic subgroup for each divisor of $|H|$. Thus, $|G| = p^2$
First you need to know that $C_m \times C_n \cong C_{mn}$ when $\gcd(m,n)=1$ and also know the fundamental theorem for abelian groups.
Now, suppose that $|G|$ has at least two distinct prime divisors, say $|G| = p_1^{a_1} \cdots p_k^{a_k}$ , $k \geq 2$. Let $p_i$ be one of the primes and let $P_i$ be the subgroup of order $p_i^{a_i}$ of $G$. Since $|P_i| < |G|$, $P_i$ is a proper subgroup of $G$ so it is cyclic. The same holds for all $i$ though, and since $G \cong P_1 \times \cdots \times P_k$, it follows that $G$ is the direct product of cyclic groups of coprime orders, so it is cyclic itself by the first observation.
Suppose now that $|G|=p^m$ for some positive integer $m$. If $m>2$, I argue that $G$ must be cyclic. By the fundamental theorem, there are positive integers $a_1, a_2, \ldots, a_n$ such that $G = C_{p^{a_1}} \times \ldots \times C_{p^{a_n}}$, where $a_1 + \ldots a_n = m$. Suppose for a contradiction that $G$ is not cyclic, but every proper subgroup of $G$ is. Then certainly $n>1$. Let $H$ be the subgroup of order $p$ of $C_{p^{a_1}}$, $K$ the subgroup of order $p$ of $C_{p^{a_2}}$. Consider the subgroup $H \times K$ of $G$. Since $|H \times K| = p^2 <|G|$, it follows that $H \times K = C_p \times C_p$ is a proper subgroup of $G$ and thus must be cyclic by assumption. But this is a contradiction, so $G$ is cyclic, just as we wanted to show.
Now, if $m=1$ there is nothing to do ($C_p$ is cyclic and its only proper subgroup is the trivial group). If $m=2$, the structure theorem tells you that either $G \cong C_{p^2}$ or $G \cong C_p \times C_p$. Note
that, in this case, the claim does not hold, since $C_p \times C_p$ is not cyclic but every proper subgroup of $C_p \times C_p$ is.
Best Answer
Suppose $G$ is not a $p$-group, then any Sylow-$p$ subgroup of $G$ is a non-trivial (since $[G:H] = p$) proper subgroup of $G$. By assumption, $H$ contains this Sylow-$p$ subgroup, but this contradicts the fact that $[G:H] = p$.