We make use of the identity
$$ \sum_{n=-\infty}^{\infty} \frac{1}{a^{2} - (x + n\pi)^{2}} = \frac{\cot(x+a) - \cot(x-a)}{2a}, \quad a > 0 \text{ and } x \in \Bbb{R}. $$
Then for $\alpha, \beta > 0$ it follows that
\begin{align*}
I := \mathrm{PV}\int_{0}^{\infty} \frac{\log\cos^{2}(\alpha x)}{\beta^{2} - x^{2}}
&= \frac{1}{2} \mathrm{PV} \int_{-\infty}^{\infty} \frac{\log\cos^{2}(\alpha x)}{\beta^{2} - x^{2}} \, dx \\
&= \frac{\alpha}{2} \mathrm{PV} \int_{-\infty}^{\infty} \frac{\log\cos^{2}x}{(\alpha\beta)^{2} - x^{2}} \, dx \\
&= \frac{\alpha}{2} \sum_{n=-\infty}^{\infty} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \frac{\log\cos^{2}x}{(\alpha\beta)^{2} - (x+n\pi)^{2}} \, dx \\
&= \frac{\alpha}{2} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \left( \sum_{n=-\infty}^{\infty} \frac{1}{(\alpha\beta)^{2} - (x+n\pi)^{2}} \right) \log\cos^{2}x \, dx \\
&= \frac{1}{4\beta} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} (\cot(x+\alpha\beta) - \cot(x-\alpha\beta)) \log\cos^{2}x \, dx,
\end{align*}
where interchanging the order of integration and summation is justified by Tonelli's theorem applied to the summation over large indices $n$. Then
\begin{align*}
I
&= \frac{1}{4\beta} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} (\cot(x+\alpha\beta) - \cot(x-\alpha\beta)) \log\cos^{2}x \, dx \\
&= \frac{1}{2\beta} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} (\cot(x+\alpha\beta) - \cot(x-\alpha\beta)) \log\left|2\cos x\right| \, dx \tag{1}
\end{align*}
Here, we exploited the following identity to derive (1).
$$ \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \cot(x+a) \, dx = 0 \quad \forall a \in \Bbb{R}. $$
Now with the substitution $z = e^{2ix}$ and $\omega = e^{2i\alpha\beta}$, it follows that
\begin{align*}
I
&= \frac{1}{2\beta} \Re \mathrm{PV} \int_{|z|=1} \left( \frac{\bar{\omega}}{z - \bar{\omega}} - \frac{\omega}{z - \omega} \right) \log(1 + z) \, \frac{dz}{z}. \tag{2}
\end{align*}
Now consider the following unit circular contour $C$ with two $\epsilon$-indents $\gamma_{\omega,\epsilon}$ and $\gamma_{\bar{\omega},\epsilon}$.
Then the integrand of (2)
$$ f(z) = \left( \frac{\bar{\omega}}{z - \bar{\omega}} - \frac{\omega}{z - \omega} \right) \frac{\log(1 + z)}{z} $$
is holomorphic inside $C$ (since the only possible singularity at $z = 0$ is removable) and has only logarithmic singularity at $z = -1$. So we have
$$ \oint_{C} f(z) \, dz = 0. $$
This shows that
\begin{align*}
I
&= \frac{1}{2\beta} \Re \lim_{\epsilon \downarrow 0} \left( \int_{-\gamma_{\omega,\epsilon}} f(z) \, dz + \int_{-\gamma_{\bar{\omega},\epsilon}} f(z) \, dz \right) \\
&= \frac{1}{2\beta} \Re \left( \pi i \mathrm{Res}_{z=\omega} f(z) + \pi i \mathrm{Res}_{z=\bar{\omega}} f(z) \right) \\
&= \frac{1}{2\beta} \Re \left( - \pi i \log(1 + \omega) + \pi i \log(1 + \bar{\omega}) \right) \\
&= \frac{\pi}{\beta} \arg(1 + \omega)
= \frac{\pi}{\beta} \arctan(\tan (\alpha \beta)).
\end{align*}
In particular, if $\alpha\beta < \frac{\pi}{2}$ then we have
$$ I = \pi \alpha. $$
But due to the periodicity of $\arg$ function, this function draws a scaled saw-tooth function for $\alpha > 0$. Of course, $I$ is an even function of both $\alpha$ and $\beta$, so the final result is obtained by even extension of this saw-tooth function.
Note that the $\csc^2(x)$ term can be expressed in terms of $\tan\left(\frac x2\right)$ aswell. To be precise we got that
\begin{align*}
\csc^2(x)=\left(\frac1{\sin(x)}\right)^2=\left(\frac1{2\sin\left(\frac x2\right)\cos\left(\frac x2\right)}\right)^2=\left(\frac{\frac1{\cos^2\left(\frac x2\right)}}{2\frac{\sin\left(\frac x2\right)}{\cos\left(\frac x2\right)}}\right)^2
=\left(\frac{1+\tan^2\left(\frac x2\right)}{2\tan\left(\frac x2\right)}\right)^2
\end{align*}
Using this and further noticing that $\frac{\mathrm d}{\mathrm dx}\tan\left(\frac x2\right)=\frac12\left(1+\tan^2\left(\frac x2\right)\right)$ we may enforce the substition $\tan\left(\frac x2\right)=u$ to obtain
\begin{align*}
I_n=\int_0^{\pi/2}\tan^n\left(\frac x2\right)\csc^2(x)\mathrm dx&=\int_0^{\pi/2}\tan^n\left(\frac x2\right)\left(\frac{1+\tan^2\left(\frac x2\right)}{2\tan\left(\frac x2\right)}\right)^2\mathrm dx\\
&=\frac12\int_0^{\pi/2}\tan^{n-2}\left(\frac x2\right)\left(1+\tan^2\left(\frac x2\right)\right)\left[\frac12\left(1+\tan^2\left(\frac x2\right)\right)\mathrm dx\right]\\
&=\frac12\int_0^1 u^{n-2}(1+u^2)\mathrm du\\
&=\frac12\left[\frac{u^{n-1}}{n-1}+\frac{u^{n+1}}{n+1}\right]_0^1\\
&=\frac12\left[\frac1{n-1}+\frac1{n+1}\right]
\end{align*}
$$\therefore~I_n~=~\int_0^{\pi/2}\tan^n\left(\frac x2\right)\csc^2(x)\mathrm dx~=~\frac n{n^2-1}$$
Best Answer
The integral is divergent if $m=0$. Assume that $m\neq 0$. With the substitution $x^m=t$, as you proposed, we get $$ \int_0^{ + \infty } {\cos (x^m )x^n dx} = \frac{1}{m}\int_0^{ + \infty } {\cos (t)t^{(n + 1)/m - 1} dt} . $$ Using the known Mellin transform formula for the cosine, we then find $$ \frac{1}{m}\int_0^{ + \infty } {\cos (t)t^{(n + 1)/m - 1} dt}= \frac{1}{m}\Gamma \!\left( {\frac{{n + 1}}{m}} \right)\cos \left( {\frac{{n + 1}}{{2m}}\pi } \right) $$ provided that $0<\frac{n+1}{m}<1$.