They do not seem incompatible to me since they talk about different types of 'totality'. The first definition takes 2 sets $X, Y$. While the second definition uses only one set. (It's a binary relation over one set, wikipedia speaks of endorelation)
You could transform the first definition so that it uses one set:
A relation $R \subset X\times X$ is total if it associates to every $a \in X$ at least one $b \in X$; that is
$$\forall a \in X, \exists b \in X: (a,b) \in R$$
However, this is weaker than the second definition. Wikipedia would speak of left-total.
Roughly said:
The first definition demands that from every element in the source at least one relation departs.
While the second definition demands that every element in a set has a connection with every other element in either one, or another direction (or both)
Compare following examples:
In the first picture the relation is left-total from $X$ to $X$. (Every element has at least one arrow departing - C has even two arrows departing). While the (endo)relation is not total as in definition 2. In the second picture it is total as in in definition 2. It also seems to be left-total.
(When a relation is total as in definition 2, it does not have to be left-total. Can you find an example?)
This
This is quite intuitive to see (e.g., a rotation of $\pi/3$ about the point $(1,1)$, would be simply $\tau \circ f$. where $f$ is a rotation of $\pi/3$ about the point $(0,0)$ and $\tau$ is a translation which takes the origin to $(1,1)$.
is not quite right. Rotation about the point $(1,1)$ fixes the point $(1,1)$. So what you actually need is $\tau \circ f \circ \tau^{-1}$. That is, drag the point $(1,1)$ to the origin, rotate, then drag it back to $(1,1)$.
Of course, it should be easy to see that
$$ (\tau \circ f \circ \tau^{-1})^n = \tau \circ f^n \circ \tau^{-1} \text{,} $$
by observing all the cancellation.
Best Answer
The list of planar symmetry groups contains the $17$ wallpaper groups ($2$-dimensional crystallographic groups), but also $7$ frieze groups and $2$ families of rosette groups in addition.
The symmetry groups listed are the classes of discrete symmetry groups of the Euclidean plane. The crystallographic ones are cocompact, i.e., $\Bbb R^2/\Gamma$ is compact, where $\Gamma$ is a discrete subgroup of $Isom(\Bbb R^2)$. These are called wallpaper groups and the classification up to abstract isomorphism (or conjugacy in the affine group) yields $17$ different groups.