First of all, I'm not sure I understand what (mod X) is, as I'm only familiar with the modulo operation that doesn't contain parantheses. However, I believe I've described how it works below:
$$ a \equiv b \pmod H \implies a\equiv r \bmod H, \quad b \equiv r \bmod H $$
Basically, a and b have the same remainder when divided by the modulus.
However, this setting is a bit different. Here, the modulus is a group (specifically a sub-group). Now, how a group can be a modulus is unknown to me, but I suspect it may be the group's order that is the modulus. Furthermore, this site stated something and (to my knowledge) didn't prove it.
$$H \subset G, \quad a,b \in H$$
$$a \equiv b \pmod H \ \ \text{if} \ \ ab^{-1} \in H$$
First of all, there is no if:
If $\ b \in H \ \ $ then $ \ \ b^{-1} \in H$ [inverse]
If $\ a,b^{-1} \in H \ \ $ then $ \ \ ab^{-1} \in H$ [closure]
That means the above statement is not a conditional one, but a necessity. As such, it can be generalized to this:
$$\forall a,b, \in G, a\equiv b \pmod G$$
So, every element in a group is congruent to any element within that group, when using that group as a modulus? If this is correct, I would like a proof and an explanation for how a group can be a modulus. If this is incorrect, I'd like to know what is actually meant by the statement from the article.
Best Answer
It looks like they are defining a relation on group $G$ using the subgroup $H$. It tells that $a,b$ are related if $ab^{-1} \in H$ and this is denoted as $a \equiv b \pmod H$.
When they define $\equiv$ it is written as
Mostly here they mean $a,b \in G$ instead of $H$ (may be a typo) because if $a,b \in H$ then already $ab^{-1} \in H$.Also see that in further equations $a,b$ are taken from $G$.