I want to decompose the tensor product $V\otimes V$, where $V=\Bbb C^5$ denotes the defining representation of $\mathrm{SO}(5)$, into irreducible representations using the following formula for the Littlewood-Richardson coefficients (see formula (15.23) of
Fuchs, Jürgen; Schweigert, Christoph, Symmetries, Lie algebras and representations. A graduate course for physicists, Cambridge Monographs on Mathematical Physics. Cambridge: Cambridge University Press. xxi, 438 p. (1997). ZBL0923.17001.):
\begin{align*}
\operatorname{mult}(\Lambda_i, V\otimes V)=\sum_{w\in W}\operatorname{sgn}(w)\operatorname{mult}_{V}(w(\Lambda_i+\rho)-\rho-\Lambda)
\end{align*}
where $W$ denotes the Weyl group of $\mathfrak g$, $\Lambda_i$ is the highest weight of some irreducible representation of $G$, $\Lambda$ denotes the highest weight of $V$ and $\operatorname{mult}_V(\nu)$ denotes the multiplicity of the weight $\nu$ in $V$. Using the standard Cartan subalgebra (see below) we have that the weights of $V$ are given by $\pm e_j,\ j=1,2$ (each with multiplicity one) with highest weight $\Lambda=e_1$.
Now we have
\begin{align*}
w(\Lambda_i+\rho)-\rho-\Lambda=\pm e_j\Leftrightarrow \Lambda_i=w^{-1}(\rho+\Lambda\pm e_j)-\rho.
\end{align*}
Since $\rho+\Lambda=\frac52 e_1+\frac12 e_2$ we get $\rho+\Lambda\pm e_1=\frac72e_1+\frac12 e_2,\ \rho$ respectively and $\rho+\Lambda\pm e_2=\frac52e_1+\frac32 e_2,\ \frac52e_1-\frac12e_2$. The first three are already in the fundamental Weyl chamber, so $w=1$ in these cases and $\Lambda_i=2e_1,0$ and $e_1+e_2$ in these cases. The last one ($\frac52e_1-\frac12e_2$) does not lie in the fundamental chamber, but choosing $w$ as the reflection along $e_2$ we get $\frac52e_1+\frac12e_2$ which does lie in the fundamental chamber. In this case we get $\Lambda_i=e_1$ occuring with multiplicity $\operatorname{sgn}(w)=-1$ which doesn't make any sense to me. Actually, the first three components $2e_1,0$ and $e_1+e_2$ already span the whole tensor product (calculating dimensions). What am I doing wrong with the last component? Thanks for any hints in advance!
*
Let $G:=\mathrm{SO}(5)$ with complexified Lie algebra $\mathfrak g^{\Bbb C}=\mathfrak{so}(5,\Bbb C)$ and standard Cartan subalgebra $\mathfrak{h}$ consisting of matrices $H=\operatorname{diag}(H_1,H_2,0)$ where $H_j=\begin{pmatrix}
0&ih_j\\
-ih_j&0
\end{pmatrix}$ for some $h_j\in\Bbb C$. Denoting by $e_j$ the functional $e_j(H)=h_j$ we obtain the root system $\Delta=\{\pm e_1\pm e_2\}\cup\{\pm e_1,\pm e_2\}$ and choose the positive system $\Delta^+:=\{e_1-e_2,e_1+e_2,e_1,e_2\}$ with half sum $\rho=\frac32 e_1+\frac12 e_2$. *
Best Answer
Besides $\pm e_i$, the original $V$ contains a fifth weight (of multiplicity $1$), namely $0$. Then for $\Lambda_i = e_1$ there are two $w\in W$ where non-trivial summands appear, namely, besides the reflection $w_2 := s_{e_2}$ you note, we also have for $w_1=id$ that $\operatorname{mult}_{V}(w_1(\Lambda_i+\rho)-\rho-\Lambda) = \operatorname{mult}_{V}(0)=1$. Since for all other $w \in W$, the weights in the formula do not occur in $V$, we have
$$\operatorname{mult}(e_1, V\otimes V)=\sum_{w\in W}\operatorname{sgn}(w)\operatorname{mult}_{V}(w(e_1+\rho)-\rho-\Lambda) = 1 +(-1) =0,$$
which is what you want because you say your other constituents $2e_1, e_1+e_2$ and $0$ already add up to the right dimension (incidentally, what is your method to compute those dimensions?).