See this answer and this one for general advice and discussion on induction. Your argument above is, alas, neither very clear nor correct.
So, we have that $|\epsilon|=0$, and that if $x=au$ with $a\in\Sigma$, $u\in\Sigma^*$, then $|x|=1+|u|$. We want to prove that for all $x,y\in\Sigma^*$, we have $|xy|=|x|+|y|$.
What you really need here is an explicit definition of $\Sigma^*$, because you need to be able to handle elements of $\Sigma^*$. I'm guessing, pending you posting the definition, that you have a certain "alphabet" $\Sigma$, and then define $\Sigma^*$ as the collection of all strings of letters from $\Sigma$ in some way.
Assuming the intended definition is the definition in this page, we can proceed as follows.
For all $n$, if $y\in \Sigma^*$ and $x\in\Sigma^n$, then $|xy|=|x|+|y|$.
If we can prove this, it will follow that for all $x$ in $\cup\Sigma^n = \Sigma^*$, we have $|xy|=|x|+|y|$, which is what we want to prove.
We will prove the proposition above by induction on $n$.
Base. $n=0$. Let $x\in\Sigma^0$. Then $x=\epsilon$, so $xy = y$ and $|x|=0$ by definition of $|\epsilon|$. Therefore, $|xy|=|y|=0+|y| = |x|+|y|$, so the equality holds.
Inductive step. We want to prove that if it is true that for all $z\in \Sigma^k$, $|zy|=|z|+|y|$, then it is true that for all $x\in\Sigma^{k+1}$, we also have $|xy|=|x|+|y|$.
Induction hypothesis. If $z\in\Sigma^k$, then $|zy|=|z|+|y|$.
Let $x\in\Sigma^{k+1}$. We want to prove that $|xy|=|x|+|y|$.
Since $x\in\Sigma^{k+1}$, then $x=au$ with $a\in\Sigma$ and $u\in\Sigma^k$.
Then we have by the definition of $|\cdot|$ that
$$|xy| = |(au)y| = |a(uy)| = 1 + |uy|.$$
And by the Induction Hypothesis, since $|u|\in\Sigma^k$, we get
$$|xy| = 1+|uy| = 1+(|u|+|y|) = (1+|u|)+|y|.$$
But by the definition of $|\cdot|$ we have $|x| = |au| = 1+|u|$, so
$$|xy| = (1+|u|)+|y| = |x|+|y|,$$
as desired.
Thus, if for every $z\in\Sigma^k$ we have $|zy|=|z|+|y|$, then for every $x\in\Sigma^{k+1}$ we have $|xy|=|x|+|y|$.
By induction, we conclude that for all $n$, if $x\in\Sigma^n$ then $|xy|=|x|+|y|$.
Therefore, we have that for all $x\in\mathop{\cup}\limits_{n=0}^{\infty}\Sigma^n$, $|xy| = |x|+|y|$.
Thus, for all $x\in \Sigma^*$, $|xy|=|x|+|y|$.
Since $y\in\Sigma^*$ is arbitrary, we conclude that
For all $x,y\in \Sigma^*$, $|xy|= |x|+|y|$
as claimed. QED
Use strong induction on the length of the string: while trying to prove this for balanced strings of some length $n>0$, you may assume the result for balanced string for any length shorter than $n$.
Now a balanced string is either empty, or of the form $(S)T$ where $S$ and $T$ are balanced strings (to see this, the first character of a non-empty balanced string must be '(', find its matching ')' which must exist, and then you've got your form $(S)T$, since both in between and after the ')' there must be balanced strings). Since both $S$ and $T$ are strictly shorter than $(S)T$, your induction hypothesis applies to them. It easily follows that the claimed result follows for $(S)T$ as well.
Best Answer
Here is a sketch that should help you to write the rigorous proof.
If $|x|=|y|$ then you get immediately $x=y$ and it's over. Otherwise, you can assume without loss of generality that $|x|<|y|$.
Since they commute, you know that $x$ is a prefix of $y$, so $y=xw$ for some $w$. But now, you can write $xwx=xxw$, so $xw=wx$. You end up with the same problem for smaller words, so you can conclude by induction.