By using some more powerful results, it is possible to do this a lot easier.
The two main ingredients for this will be the following:
Burnside's $pq$-Theorem: If only two distinct primes divide the order of $G$, then $G$ is solvable.
Burnside's Transfer Theorem: If $P$ is a $p$-Sylow subgroup of $G$ and $P\leq Z(N_G(P))$ then $G$ has a normal $p$-complement.
So when looking for a non-abelian simple group, the first result immediately tells us that we can assume at least $3$ distinct primes divide the order of $G$.
The way we will use the second result is the following:
Let $P$ be a $p$-Sylow subgroup where $p$ is the smallest prime divisor of $|G|$. We will show that if $|P| = p$ then $P\leq Z(N_G(P))$ (and then the above result says that $G$ is not simple).
To see this, we note that $N_G(P)/C_G(P)$ is isomorphic to a subgroup of $\rm{Aut}(P)$ (this is known as the N/C-Theorem and is a nice exercise), which has order $p-1$, and since the order of $N_G(P)/C_G(P)$ divides $|G|$, this means that $N_G(P) = C_G(P)$ and hence the claim (since we had picked $p$ to be the smallest prime divisor).
The same argument actually shows that if $p$ is the smallest prime divisor of $|G|$, then either $p^3$ divides $|G|$ or $p = 2$ and $12$ divides $|G|$. The reason for this is that if the $p$-Sylow is cyclic of order $p^2$ the precise same argument carries through, since in that case all prime divisors of $|\rm{Aut}(P)|$ are $p$ or smaller.
If $P$ is not cyclic then the order of $\rm{Aut}(P)$ will be $(p^2-1)(p^2 - p) = (p+1)p(p-1)^2$ and the only case where this can have a prime divisor greater than $p$ is when $p = 2$ in which case that prime divisor is $3$, and we get the claim.
In summary we get that the order of $G$ must be divisible by at least $3$ distinct primes, and either the smallest divides the order $3$ times, or the smallest prime divisor is $2$ (actually, by Feit-Thompson, we know this must be the case, but I preferred not to also invoke that), and $3$ must also divide the order.
This immediately gives us $60$ as a lower bound on the order of $G$, and the next possible order would be $2^2\cdot 3\cdot 7 = 84$ and all further orders are greater than $100$. So we are left with ruling out the order $84$ (which you have done), and showing that the only one of order $60$ is $A_5$.
Answer with hints that must be checked and solved:
As already proved, a group $\;G\;$ of order $\;30\;$ must have either one unique subgroup of order five or one unique subgroup of order three.
Take now the unique such subgroup, say $\;N\;$ , and one of the (possibly several) groups of the other order, say $\;H\;$ . Since clearly $\;N\cap H=1\;$ , we get that $\;NH\;$ is a subgroup (why?) of order $\;15\;$ and thus $\;NH\lhd G\;$ (why?).
But a group of order $\;15\;$ is necessarily cyclic (why?), and since any subgroup of a normal cyclic subgroup of a big group is normal in the big group (why?), we get that both $\;N,H\lhd G\;$
Best Answer
We may as well assume $p < q$. The number of Sylow $q$-subgroups is $1$ mod $q$ and divides $p^2$. So it is $1, p$, or $p^2$. We win if it's $1$ and it can't be $p$, so suppose it's $p^2$. But now $q \mid p^2 - 1$, so $q \mid p+1$ or $q \mid p-1$.
Thus $p = 2$ and $q = 3$. The case of order $36$ has been proved in an earlier question:
No group of order 36 is simple