The mass density varies as the distance from the x-axis implies that $\rho =Ky$ where $K$ is a constant.
Now, the total mass is given by
$$M=\int_0^1 \int_y^{\sqrt{y}} (Ky) dx dy$$
The moment about x is
$$\frac{\int_0^1 \int_y^{\sqrt{y}} x(Ky) dx dy}{M}$$
$$\frac{\int_0^1 \int_y^{\sqrt{y}} y(Ky) dx dy}{M}$$
Can you complete? Notice that the moments are independent of $K$.
I am doing this in two stages. The disc needs to be divided into elements in the form of infinitesimally thin and concentric hoops, which we then integrate. But first we have to find the MI of a hoop about an axis at angle $\frac {\pi}{6}$ to the plane of the hoop, through the centre of the hoop.
Consider a circle of radius $r$ in the $xy$ plane, centre $O$, and a point on the circumference with coordinates $P(r\cos\theta,r\sin\theta, 0)$.
We will write the axis, as specified in the question, as the line $$\underline{r}=t\left(\begin{matrix}\frac{\sqrt{3}}{2}\\0\\\frac 12\end{matrix}\right)$$
The distance $d$ from $P$ to the line is given by the magnitude of the cross product $$\overrightarrow{OP}\times\underline{\hat{r}}$$
A quick calculation gives $$d^2=\frac 14r^2(1+3\sin^2\theta)$$
Now let the circle be a hoop of mass $m$ and density per unit length $\rho$
The MI of an element of mass $\delta m$ at $P$ is $\delta I=\delta m d^2=\rho r \delta\theta d^2$
Therefore for the whole hoop, $$I=\frac 14\rho r^3\int_0^{2\pi}(1+3\sin^2\theta)d\theta=\frac 58mr^2$$
Now finally we can consider the disc of radius$R$, mass $m$ and density per unit are $\rho$ divided into concentric elements of thickness $\delta x$ and radius $x$
Then $$\delta I=\frac 58\delta mx^2=\frac 54\pi\rho x^3\delta x$$
$$\Rightarrow I=\frac 54\pi\rho\int_0^Rx^3dx$$
$$\Rightarrow I=\frac{5}{16}mR^2$$
Best Answer
The problem is in how you wrote your angular integral. It is not $$2\int_0^\alpha...,$$ but $$\int_{-\alpha}^{\alpha}...$$ Then you will have $\cos\alpha-\cos(-\alpha)=0$.