I believe the comment by James is correct, these groups are precisely the $2$-Engel groups.
Claim: The following statements are equivalent for a group $G$.
Every centralizer in $G$ is a normal subgroup.
Any two conjugate elements in $G$ commute, ie. $x^g x = x x^g$ for all $x, g \in G$.
$G$ is a $2$-Engel group, ie. $[[x,g],g] = 1$ for all $x, g \in G$.
Proof:
1) implies 2): $x \in C_G(x)$, thus $x^g \in C_G(x)$ since $C_G(x)$ is normal.
2) implies 3): $x^g = x[x,g]$ commutes with $x$, thus $[x,g]$ also commutes with $x$.
3) implies 1): If $[[x,g],g] = 1$ for all $g \in G$, then according to Lemma 2.2 in [*], we have $[x, [g,h]] = [[x,g],h]^2$. Therefore $[C_G(x), G] \leq C_G(x)$, which means that $C_G(x)$ is a normal subgroup.
[*] Wolfgang Kappe, Die $A$-Norm einer Gruppe, Illinois J. Math. Volume 5, Issue 2 (1961), 187-197. link
Let $N$ be any finite solvable group equipped with an action of a cyclic group $C=\{1,a\}$ of order $2$ which fixes no elements besides the identity. For instance, $N$ could be any abelian group of odd order (not necessarily cyclic) with $a$ acting by negation. Let $G=N\rtimes C$ be the semidirect product given by this action. Then $G$ is solvable, $N$ is a normal subgroup, and I claim that $G\setminus N$ is a single conjugacy class. Indeed, note that the element $(1,a)\in G$ does not commute with $(x,1)$ for any $x\neq 1$ in $N$ (since $a$ acts nontrivially on every non-identity element of $N$). It follows that the centralizer of $(1,a)$ is just $\{(1,a),(1,1)\}$, and so the conjugacy class of $(1,a)$ contains $|G|/2$ elements. But this conjugacy class is a subset of $G\setminus N$ which also has $|G|/2$ elements, so the conjugacy class must be all of $G\setminus N$.
Conversely, all finite examples are of this form: suppose $G$ is a finite group with a normal subgroup $N$ such that $G\setminus N$ is a conjugacy class. Picking any $a\in G\setminus N$, we then have $|G\setminus N|=|G|/|C_G(a)|$. In particular, since $|C(x)|\geq 2$, we have $|G\setminus N|\leq |G|/2$ and so $|N|\geq|G|/2$. This can only happen if $|N|=|G|/2$, so $N$ must have index $2$ and $C_G(a)$ must have order $2$.
This means that $a$ must be an element of $G$ of order $2$ which commutes with no elements of $G$ except itself and $1$. Moreover, we see that $G$ is the semidirect product of the subgroup $C$ generated by $a$ and $N$, and $a$ must act nontrivially on every non-identity element of $N$ since $a$ commutes with nothing except itself and $1$.
Best Answer
The coset space is always a homogeneous compact Hausdorff space even if it is not a group, and that is enough to conclude it is either finite or uncountable. If it is not discrete, then by homogeneity it has no isolated points, and a nonempty compact Hausdorff space with no isolated points is uncountable.