This is a question that touches on many issues. On the one hand, things are indeed easier to deal with in the language of the complexification. For $SL(2,\mathbb{C})$, there is a specific element $H$ in the Lie algebra $\mathfrak{sl}(2,\mathbb{C})$, which acts diagonalizably on any finite dimensional representation. The eigenvalues are called weights, and in the case of an irreducible representation, they are all different. Specifically, the $n$-dimensional irreducible representation $\mathbf{n}$ admits a basis of weight vectors of weight $-n+1,-n+3,\dots,n-3,n-1$. Now in a tensor product, the tensor product of two eigenvectors for $H$ is an eigenvector with eigenvalue the sum of the eigenvalues of the two factors. There is a general result saying that the weight vector with highest possible weight will always generate an irreducible subrepresentation, and there there is an invariant complement to this subrepresentation. This allows you to compute decompositions in an elementary way.
Take the example of $\mathbf{3} \otimes \mathbf{3}$. In $\mathbf{3}$ you have weights $-2$, $0$, and $2$, so in the tensor product, the possible weights are $-4$, $-2$, $0$, $2$, and $4$ and the dimensions of the eigenspaces are $1$, $2$, $3$, $2$, and $1$. The weight vector of weight $4$ generates a subrepresentation $\mathbf{5}$, which has one weight vector for each of the listed weights. So in the complement, you get weights $-2$, $0$, and $2$ with dimensions $1$, $2$, and $1$. The highest of these gives you a subrepresentation $\mathbf{1}$ again including one of each of the weights, so there is just one copy of $\mathbf{0}$ left. To describe the result in terms of symmetry, you observe that the weight vector of the maximal weight $4$ is the tensor product of a highest weight vector with itself, so this sits in $S^2\mathbf{3}$. Either counting dimensions or by direct analysis of the weights you can see that $S^2\mathbf{3} \cong \mathbf{5} \oplus \mathbf{1}$ and $\Lambda^2\mathbf{3} \cong \mathbf{3}$.
This works similarly for general tensor products $\mathbf{n} \otimes \mathbf{m}$ for $n \geq m$. This is isomorphic to $\mathbf{n+m-1} \oplus \dots \oplus \mathbf{n-m+1}$ with dimension decreasing by two in each step (so there are always $m$ summands). For $n=m$, things look similar as in the special case, $S^2\mathbf{n} = \mathbf{2n-1} \oplus \mathbf{2n-5} \oplus \dots$ and $\Lambda^2\mathbf{n} = \mathbf{2n-3} \oplus \mathbf{2n-7} \oplus \dots$ with dimensions going down in steps of $4$.
You can always construct higher tensor powers step by step, say $\mathbf{2}\otimes \mathbf{2} \otimes \mathbf{2} \cong (\mathbf{3} \oplus \mathbf{1})\otimes \mathbf{3} \cong (\mathbf{3} \otimes \mathbf{2}) \oplus \mathbf{2}$ and then the first summand splits as $\mathbf{4} \oplus \mathbf{2}$. However, this is just one possible way to “decompose according to symmetry”, since one has distinguished the first two factors. In fact, the canonical way to decompose higher tensor products is just into so-called isotypical components. Here this would be $\mathbf{2} \otimes \mathbf{2} \otimes \mathbf{2} \cong \mathbf{4} \oplus W$, where $W$ is an invariant subspace isomorphic to a direct sum of two copies of $\mathbf{2}$. However, there are various possible realizations for this isomorphism, none of which is canonical. the systematic way to deal with this is simultaneously decomposing as a representation of $\mathfrak{sl}(2,\mathbb{C})$ and the permutation group $\mathfrak{S}_3$. This indeed leads towards Young diagrams, which are also needed to deal with $SL(n,\mathbb{C})$ and hence with $SU(n)$. If you are looking for literature in that direction, I would recommend the book of Fulton and Harris on representation theory.
Best Answer
Denote $S^{Tr}$ to be the space of symmetric traceless tensors. Let $D\subset S^{Tr}$ be a non-trivial subspace of $S^{Tr}$ that is irreducible with respect to $SO(3)$. We will prove $D = S^{Tr}$.
$D$ is closed with respect to $SO(3)$ transformations. Specifically: $\forall \; T^{ab} \in D, \; R_{x}^{a} \in SO(3)$, we have $R_{x}^{a}T^{xy}R_{y}^{b}\in D$.
If $T^{ab}\in D$ then $T^{ab}\in S^{Tr}$. Also, if $\;T_{1}^{ab}, T_{2}^{ab}\dots T_{n}^{ab} \in D$ then $\text{span}\{T_{1}^{ab}, T_{2}^{ab}\dots T_{n}^{ab}\} \subset D$.
We must show that $\forall T\in S^{Tr}$ with $T\neq 0\;$, $\exists R_{1},R_{2} \dots R_{n}\in SO(3)$ such that $\text{span}\{R_{1x}^{a}T^{xy}R_{1y}^{b}, R_{2x}^{a}T^{xy}R_{2y}^{b}\dots R_{nx}^{a}T^{xy}R_{ny}^{b}\} = S^{Tr}$. In short, any non-zero tensor of $S^{Tr}$ can generate a basis for $S^{Tr}$ under the action of $SO(3)$, thus proving the ireductibility .
From now it's a problem of linear algebra. A symmetric traceless tensor $T^{ab}$ can be bought in the form $\left(\begin{smallmatrix} a & 0 & 0\\ 0 & b & 0\\ 0 & 0 & -(a+b) \end{smallmatrix}\right)$ by a suitable rotation matix (symetric matrices are diagnoalizable). With the rotation matrix $\left(\begin{smallmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0 \end{smallmatrix}\right)$ we bring it to the form $\left(\begin{smallmatrix} b & 0 & 0\\ 0 & -(a+b) & 0\\ 0 & 0 & a \end{smallmatrix}\right)$, with which we form a basis for all traceless diagonal tensors. By linear combinations we obtain the matrix $\left(\begin{smallmatrix} 1 & 0 & 0\\ 0 & -1 & 0\\ 0 & 0 & 0 \end{smallmatrix}\right)$, which can be transformed via suitable rotation matrices in $\left(\begin{smallmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 0 \end{smallmatrix}\right)$,$\left(\begin{smallmatrix} 0 & 0 & 1\\ 0 & 0 & 0\\ 1 & 0 & 0 \end{smallmatrix}\right)$,$\left(\begin{smallmatrix} 0 & 0 & 0\\ 0 & 0 & 1\\ 0 & 1 & 0 \end{smallmatrix}\right)$.
Starting with any traceceless symetric tensor, and applying only $SO(3)$ transformations and linear combinations, we can form a basis for $S^{Tr}$, thus obtain any possible traceceless symmetric tensor. Thus $S^{Tr}$ is irreducible with respect to $SO(3)$.