Don't forget about the "$= 0$" part of the equations.
The formula $(ax^2+ay^2+2(a'x+b'y)+c,0)$ is not an equation;
the equivalent equation to $a\lvert z\rvert^2+\bar{b}z+b\bar{z}+c=0$ is
$$(ax^2+ay^2+2(a'x+b'y)+c,0) = (0,0),$$ or more simply
$$ax^2+ay^2+2(a'x+b'y)+c = 0.$$
For the equation $\lvert c\rvert z-c\lvert z\rvert=0,$
in your $\mathbb R^2$ notation you get the equation
$$ \left(\sqrt{q^2+r^2}x-\sqrt{x^2+y^2}q,\sqrt{q^2+r^2}y-\sqrt{x^2+y^2}r\right) = (0,0).$$
That is equivalent to the two simultaneous equations
\begin{align}
x\sqrt{q^2+r^2} - q\sqrt{x^2+y^2} &= 0,\tag1 \\
y\sqrt{q^2+r^2} - r\sqrt{x^2+y^2} &= 0.\tag2
\end{align}
We have to assume $c\neq 0,$ otherwise your equation is true everywhere.
So we can divide this into two cases.
In the first case, $r = 0$ but $q \neq 0.$ In that case,
Equation $(2)$ says that $y = 0,$ and therefore Equation $(1)$ says that
$x\lvert q\rvert = q \lvert x\rvert.$
This is true for all non-negative $x$ when $q > 0$ and for all non-positive $x$ when $q < 0.$
In the second case, $r\neq 0.$In that case,
Equation $(2)$ says that
$$ \sqrt{x^2+y^2} = \frac yr \sqrt{q^2+r^2}, $$
and plugging that into Equation $(1)$ we get
$$
x\sqrt{q^2+r^2} - y \frac qr \sqrt{q^2+r^2} = 0.
$$
This is an equation of the form $ax^2+ay^2+2(a'x+b'y)+c = 0$
where $a = 0,$ $a'= \frac12\sqrt{q^2+r^2},$ $b' = -\frac q{2r} \sqrt{q^2+r^2},$
and $c = 0.$
And it would be an equation of a line through the origin, except that
Equation $(2)$ also implies that $y$ has the same sign as $r,$ and therefore you only get half of the line.
Starting over from the beginning, a general equation for an arbitrary line in $\mathbb R^2$ (expressed in $(x,y)$ coordinates) is
$$Ax + By + C = 0, \tag3$$
where the coefficients $A,$ $B,$ and $C$ are whatever they need to be in order to produce an equation of the desired line.
An equation of this form exists for any line at any angle (even parallel to the $y$ axis) through any point in the plane.
For example:
To represent the line with equation $y = 3x + 5,$
simply set $A=3,$ $B=-1,$ and $C=5,$ and then you have the equivalent equation $3x + (-1)y + 5 = 0$, which is in the form $Ax + By + C = 0.$
To represent the line $y = -7,$ we can set $A=0,$ $B=1,$ and $C=7$ to get the equivalent equation $0x + 1y + 7 = 0.$
To represent the line $x = 15,$ simply set $A=1,$ $B=0,$ and $C=-15$ so that the equation $Ax + By + C = 0$ becomes $1x + 0y + (- 15) = 0.$
In short, the equation $Ax + By + C = 0$ is a very handy one (not just for this problem!) and it is good to make it as familiar as equations such as
$y = mx + k$ and $x^2 + y^2 = r^2.$
Surely it is not hard to see how to put the equation
$ax^2+ay^2+2(a'x+b'y)+c = 0$
into the form of Equation $(3).$
Best Answer
Brian seems to have given a complete solution, but to address your specific confusion: You're right that the equation $$z(|s|^2-|t|^2) = \bar{r}t - r\bar{s}$$ will have infinitely many solutions if $|s|^2 - |t|^2 = 0$ and $\bar{r}t - r\bar{s}= 0.$ But, in that case, the elimination method step (where we solve for $z$ by eliminating the conjugate) is producing the trivial equation $0 = 0,$ which loses the original solution set. The arbitrary $z$ values you choose won't satisfy the original equation - only this secondary equation.
For example, take $s = t = \frac{1}{2}$ and $r = -1$ so that the original equation becomes $$\frac{z}{2} + \frac{\bar{z}}{2} - 1 = 0,$$ which is equivalent to $\Re(z) = 1$. Clearly that solution set is a line.
If you take the conjugate equation and use it to eliminate $\bar{z}$, then $z$ will be eliminated as well, and you'll get the equation $0z + 0\bar{z} = 0$, which does have infinitely many solutions but isn't equivalent to the original equation. We've lost the original equation's information.
The overall problem is arising because the conjugate equation is a multiple of the original equation in this degenerate case, so it's not useful for elimination. For example, if you start with the system of equations \begin{array}{11}x + y = 0\\2x + 2y = 0,\end{array} then there's a line (not plane) of solutions: $\{(x, -x): x \in \mathbb{R}\}$, but the elimination method will kill both variables and lose this information, because this is a dependent system of equations. You've identified the conditions under which the conjugate trick/elimination method fails.