You need the equivalence relation to be a congruence.
The equivalence relation $\sim$ on the monoid $M$ is called a congruence to mean that
$(x \sim a $ and $ y \sim b)\implies xy \sim ab$ and $yx \sim ba$ for all $x,y,a,b \in M$
In this case we form the group of equivalence classes and define multiplication just like for groups: We define $[x][y] = [xy]$. The relation being a congruence ensures this is well-defined.
When $M$ is a group we can prove $[1]$ is a normal subgroup, and the cosets of $[1]$ are exactly the partition elements. For monoids it's still true that $[1]$ is closed under inverses whenever they exist.
That's because $x \in [1] \implies x \sim 1 \implies x^{-1}x \sim x^{-1} \implies 1 \sim x^{-1} \implies x^{-1} \in [1]$.
But in general $[1]$ is not a normal subgroup; and the cosets don't correspond to the partition elements; and don't even form a partition!
For example consider $(\mathbb N,\times)$ under the parity relation. It's easy to see $[1] = \{1,3,5, \ldots\}$ is just the odd numbers. We might like to have the coset $2[1] =$ the even numbers. But unfortunately $2[1] = \{2,6,10, \ldots\}$. Moreover $[2] = \{2,4,6, \ldots\}$ is different to $2[1]$.
In your particular case you want to define $(x \sim y \iff x=uy$ for some invertible $u)$. This is indeed a congruence so we can define the quotient ring as above. Moreover the partition elements turn out to be what we want: $[1]$ is the normal subgroup $U$ of all units. To get the class of $x$ just take all left-multiples by units. In other words $[x] = Ux = [1]x$.
We could replace $U$ with any normal subgroup of the group of units and this would work.
It is well known (and stated in most textbooks) that a continuous surjection $f : X \to Y$ is a quotient map if and only if the following is satisfied for all functions $g : Y \to Z$:
$g$ is continuous if and only if $g \circ f : X \to Z$ is continuous.
An obvious corollary is this.
Given two quotient maps $f : X \to Y, f' : X \to Y'$ and a bijection $\phi : Y \to Y'$ such that $\phi \circ f = f'$. Then $\phi$ is a homeomorphism.
Each quotient map $f : X \to Y$ induces an equivalence relation on $X$ by defining $x \sim x'$ iff $f(x) = f(x')$. Now apply the corollary to $f$ and the quotient map $p : X \to X/\sim$.
In fact, this does not frequently occur as an explicit statement in the literature.
Best Answer
Let $M$ be a monoid and let $P$ be a nonempty subset of $M$. Define a relation $\equiv_P$ on $M$ by setting $$ x \equiv_P y \quad\text{if and only if}\quad (x \in P \iff y \in P) $$ This is an equivalence relation of index $2$, since there are exactly two equivalence classes: $(P \times P) \cup (P^c \times P^c)$ and $(P \times P^c) \cup (P^c \times P)$. Thus if $\equiv_P$ is a congruence, the quotient monoid $M/{\equiv_P}$ has two elements. There are only two monoids of size $2$: the monoid $U_1 = \{0, 1\}$ (with the usual multiplication) and the cyclic group $C_2$ of order $2$.
Let $\pi:M \to M/{\equiv_P}$ be the quotient morphism. If $P$ contains $1$, in particular if $P$ a submonoid of $M$, then $P = \pi^{-1}(1)$. It follows that
Coming back to the case of a free monoid $M = A^*$, a submonoid $P$ of $A^*$ defines a congruence if and only if $P = B^*$ (in the $U_1$ case), or, interestingly, if $P = (A^2)^*$ (in the $C_2$ case). Note that $P = (A^3)^*$ does not define a congruence, since in this case, $a \equiv_P a^2$ but $a^2 \not\equiv_P a^3$.