Let $ G $ be a finite group. Let $ \pi_i, \pi_j, \pi_k $ be irreps of $ G $. Then consider the sum
$$
f(x):= \frac{1}{|G|} \sum_{g \in G} \chi_i(xg^{-1}) \chi_j(g) \pi_k(g)
$$
If $ \pi_k $ is the trivial rep then we have the following sum
$$
\frac{1}{|G|} \sum_{g \in G} \chi_i(xg^{-1}) \chi_j(g) = \frac{1}{d_i} \langle \chi_i,\chi_j \rangle \chi_i(x)
$$
evaluated using the convolution of characters formula given here Convolution of irreducible characters of a finite group . Note that if $ \chi_i, \chi_j $ are irreducible then $ \langle \chi_i,\chi_j \rangle =\delta_{i,j} $ as in the linked question.
When $ \pi_k $ is not the trivial rep what does this sum equal?
My guess for the correct formula is
$$
f(x):= \frac{1}{|G|} \sum_{g \in G} \chi_i(xg^{-1}) \chi_j(g) \pi_k(g)= \frac{1}{d_id_k} \langle \chi_i , \chi_j\chi_k \rangle \chi_i(x) \pi_k(1)
$$
Note that if $ \pi_k $ is the trivial rep then $ \langle \chi_i ,\chi_j \chi_k \rangle = \langle \chi_i, \chi_j \rangle $ so this reduces to the character convolution formula given above.
Response to comment: Let $ G \cong SL(2,5) $ be a subgroup of $ SU(2) $. The only example I've seen with the scalar product $ \langle \chi_i ,\chi_j \chi_k \rangle \neq 0 $ is the case of $ \pi_2 $ (the restriction of the unique 2d irrep of $ SU(2) $ to $ G $) and $ \pi_3 $ (the restriction of the unique 3d irrep of $ SU(2) $ i.e. the adjoint rep). In this case $ \langle \chi_2,\chi_2 \chi_3 \rangle=1 $. If we consider the sum
$$
\frac{1}{|G|} \sum_{g \in G} \chi_2(xg^{-1}) \chi(g) \pi_3(g)
$$
and think of $ \pi_3 $ as actually being the adjoint representation $ [\pi_3(g)]M:=gMg^{-1} $ where $ M $ is a traceless $ 2 \times 2 $ matrix, then it seems that something like this is true
$$
\frac{1}{120} \sum_{g \in G} \chi_2(xg^{-1}) \chi(g) gMg^{-1}=\frac{1}{6} tr(x)M+ \frac{1}{6}[M,x]
$$
I think the meaning of the $ 1/6 $ factor is just that $ 1/6=1/d_id_k=1/(2)(3) $. The $ tr(x)M $ term just multiplies by a scalar but the commutator term $ [M,x] $ is more puzzling.
Best Answer
First, I want to establish a character convolution formula. Suppose we index our irreps $\pi_0,\pi_1,\cdots$ and corresponding characters $\chi_i$ and degrees $d_i$. Recall in the group algebra we have orthogonal idempotents
$$ e_i=\frac{d_i}{|G|}\sum_{g\in G}\chi_i(g^{-1})g. $$
That is, applying $e_i$ is the projection onto the $\pi_i$-isotypical component of a representation. We have the orthogonality relation $e_je_i=\delta_{ij}e_i$. The coefficient of $g^{-1}$ on both sides is
$$ \frac{d_id_j}{|G|^2}\sum_{uv=g^{-1}}\chi_j(u^{-1})\chi_i(v^{-1})=\delta_{ij}\frac{d_i}{|G|}\chi_i(g). $$
Setting $a=v^{-1}$, $b=u^{-1}$ and rescaling we can rewrite this as
$$ \frac{1}{|G|}\sum_{ab=g}\chi_i(a)\chi_j(b)=\delta_{ij}\frac{\chi_i(g)}{d_i}. $$
Now for your question. Define
$$ f(x)=\frac{1}{|G|}\sum_{g\in G}\chi_i(xg^{-1})\chi_j(g)\pi_k(g). $$
We may readily verify $\pi_k(a)f(x)\pi_k(a)^{-1}=f(axa^{-1})$. Thus, if we average $f(x)$ over a normal subset of $G$, the result ought to be an intertwiner, hence a scalar transformation, and we may find the scalar by computing the trace. To get the trace of $f(x)$, recall any class function $\psi$ has a Fourier decomposition in terms of irreducible characters: $\psi(g)=\sum_\ell\langle\psi,\chi_\ell\rangle\chi_\ell(g)$. So we may compute
$$\mathrm{tr}\,f(x)=\frac{1}{|G|}\sum_{ab=g}\chi_i(a)(\chi_j\cdot\chi_k)(b)=\frac{1}{|G|}\sum_{ab=g}\chi_i(a)\sum_\ell \langle\chi_\ell,\chi_j\chi_k\rangle\chi_\ell(b) $$
$$ =\sum_\ell\langle\chi_\ell,\chi_j\chi_k\rangle\Big(\frac{1}{|G|}\sum_{ab=g}\chi_i(a)\chi_\ell(b)\Big)=\sum_\ell\langle\chi_\ell,\chi_j\chi_k\rangle \delta_{i\ell}\frac{\chi_i(g)}{d_i}=\langle\chi_i,\chi_j\chi_k\rangle\frac{\chi_i(g)}{d_i}. $$
Thus, for any conjugacy class $X\subset G$, we have
$$\frac{1}{|X|}\sum_{x\in X}f(x)=\langle\chi_i,\chi_j\chi_k\rangle\frac{\chi_i(X)}{d_i}I. $$
It's not obvious to me at the moment what we can say about $f(x)$ without averaging.