I will unashamedly say that this was at least spurred by homework. However I have gone far beyond the syllabus of the course and still can't find an authoritative answer. And it seems an interesting question to me that I doubt the professor will answer (if I wasn't ashamed to ask).
I am asked to calculate the Fourier transform of the convolution of two signals, for generality:
$\mathrm{F}\left\lbrace\sin^3\left(at+b\right)*\cos^3\left(ct+d\right)\right\rbrace$.
I have tried two approaches.
First, take the product of the Fourier transforms of the sinusoids. This leads to an expression that contains terms of the form $\delta(\omega-a)\delta(\omega-b)$. According to [1] and unless I missed it, the product of two distributions, unlike other operations, is not defined.
Secondly calculate the convolution directly. This leads me to an integral of the form:
$\int_{-\infty}^{\infty}\sin^3\left(a\tau+b\right)\cos^3\left(c(\tau-t)+d\right)\mathrm{d}\tau$
This also I think is non-convergent.
So am I right to think that this convolution and its Fourier transform are not defined?
[1] Zemanian: Distribution Theory and Transform Analysis
Best Answer
It is not difficult to find a test function $\varphi$ so that $$ (\sin \ast \cos) (\varphi) = \int \sin x\int \cos t \varphi(t-x) dt dx $$ does not exist. Hence $sin \ast cos$ is not a (tempered) distribution, so you cannot expect the existence of its Fourier Transform. The same is true for the product of its separate Fourier Transforms.
There are specific situations in which you can try to extend either the convolution or the multiplication, but for your case I don't expect anything useful.