Find all primes $p$ and $q$ such that $p^2 + 7pq + q^2$ is a perfect square.
One obvious solution is $p = q$ and under such a situation all primes $p$ and $q$ will satisfy.
Further if $p \neq q$ then we can assume without the loss of generality that $p > q$. Assuming this and that there exists at least one such perfect square I have tried to show some contradiction modulo $4$ as any odd perfect square leaves a remainder of $1$ when divided by $4$, but it is not working. However I firmly believe that $p = q$ is the only solution, but I have failed to prove this.
Best Answer
If $p^2+q^2+7pq = r^2$ ($r$ being any integer), then $(p+q)^2 + 5pq = r^2$. So $5pq = r^2-(p+q)^2 = (r+p+q)(r-p-q)$.
Since $p, q$ and $5$ are all prime, it follows that one of the factors on the right-hand side is equal to one of them, and the other factor is the product of the other two. As clearly visible, $r+p+q$ is greater than any of the numbers $p, q$ and $5$. So it must be the product of two of those numbers (maybe three too), and the other factor $r-p-q$ must be equal to $p, q$ or $5$ (or $1$).
Now, different cases arise:
CASE $1$:
If $r-p-q = p$ then $r=2p+q$, and thus original equation becomes $p^2+q^2+7pq = (2p+q)^2$, which simplifies to $p=q$, Same if $r-p-q = q$.
CASE $2$: If $r-p-q = 5$ then $r = p+q+5$, and the equation $5pq = (r+p+q)(r-p-q)$ becomes $pq = 2(p+q)+5$. You can write this as $(p-2)(q-2)=9$, and the only solutions to this are $p=q=5$ and $p=3;q=11$ (or the other way round).
CASE $3$: $r+p+q$ might be equal to the product of all three numbers $5pq$, with $r-p-q = 1$. But then the equation becomes $2p+2q+1 = 5pq$, which is clearly impossible because the right-hand side is visibly greater than the left-hand side (Though there are solutions like $(1,1)$, but $1$ is not prime ).
So, to summarise the whole answer, We can say that only solutions are:
$(p,q)=(p,p),(3,11),(11,3)$
SOURCE