I’ll get you started. To show that $\sim$ is reflexive, you must show that if $n\in\Bbb N$, then $n\sim n$. Check the definition of $\sim$: this means that $n\cdot n$ is a square. Of course $n\cdot n=n^2$ is a square, so $n\sim n$, and $\sim$ is reflexive. You should have no trouble showing that $\sim$ is symmetric. For transitivity, suppose that $k\sim m$ and $m\sim n$. Then $km$ and $mn$ are squares, say $km=a^2$ and $mn=b^2$; you must show that $k\sim n$, i.e., that $kn$ is a square. Try to write $kn$ in terms of the pieces that you already have, doing it in a way that demonstrates that $kn$ is a square.
$[3]$ is by definition the set of all $n\in\Bbb N$ such that $3\sim n$, i.e., such that $3n$ is a square. What does this tell you about $n$? HINT: What can you say about the number of factors of $3$ in the prime factorization of $n$? Thinking in similar terms will get you through the rest of (b) as well.
For (c) you should be thinking about the prime factorizations of $a$ and $b$.
One thing that jumps out is that we are working in $\pmod 5$, and it just happens that $2+3=5$. Let's keep that in mind for now.
To show transitivity, we assume
Assume $a$ ~ $b$ and $b$ ~ $c$. Thus we assume that
$$2a+3b \equiv 0 \pmod 5$$
and
$$2b+3c \equiv 0 \pmod 5$$
We want to be able to conclude that $2a+3c \equiv 0 \pmod 5$.
Now what is the first way that comes to mind if we want $a$ and $c$ to show up in the same congruence relation? Well to add the relations we already have. This gives
$$2a+5b + 3c \equiv 0 \pmod 5$$
But $5 \equiv 0 \pmod 5$, so $5b \cong 0 \cdot b = 0 \pmod 5$, so we have
$$2a + 3c \equiv 0 \pmod 5$$
as we wanted.
EDIT:
This is the way it works with congruence relations:
You can prove the following facts (we take them for granted with usual equality, but you have to prove them for congruence):
$(1)$ If $a \equiv b \pmod n$, and $c \equiv d \pmod n$ then $a+c \equiv b+d \pmod n$ (moral: you can add two congruences together)
$(2)$ If $a \equiv b \pmod n$, then $ac \equiv bc \pmod n$ for any integer $c$. (moral: you can multiply both sides of a congruence by something)
So now:
We can prove from the definition of congruence that $-5 \cong 0 \pmod 5$. Therefore, using $(2)$, we can say that $(-5) \cdot b \equiv 0 \cdot b \pmod 5$, so $-5b \equiv 0 \pmod 5$.
Using $(2)$, we can add the congruences $2a+5b + 3c \equiv 0 \pmod 5$ and $-5b \equiv 0 \pmod 5$ to get $2a+3c \equiv 0 \pmod 5$.
This is how it's done very formally; in practice, in any congruence relation, if you see some $a$ and you know that $a \equiv b$, then you can replace $a$ with $b$ in some or all places where $a$ occurs. That's why I said in the original that $5b \equiv 0 \pmod 5$ and we are done, since you can just replace $5b$ by $0$.
A note about $(2)$: you can always multiply both sides, but you can't always divide, or cancel out common factors. For example, $4 \cdot 3 \equiv 2 \cdot 3 \pmod 6$, but $3 \not \equiv 2 \pmod 6$. Canceling of course does work in some places, but in only works in all cases if you are modulo a prime number; then it's ok to cancel like this.
Best Answer
Hint: $ac = \frac{(ab)(bc)}{b^2}$.