The moment of inertia is defined as
$$I = \int r^2 dm$$
but I am not sure how to proceed with solving the above integral. All examples I have seen seem to be done with different strategies. They usually start: "well consider this part of the mass $dm$" and then suddenly just plug in $2 \pi r $ and the answer miraculously pops out.
I can not find some (good) guidelines on how to solve the integral. What function is actually integrated, because both mass $m$ and distance $r$ can be variable in one another:
$$I = \int f(m(r),r(m)) r^2 dr$$
Best Answer
TL;DR This question is actually more related to mathematics than physics. Here I give just some basic guidelines which can help you in most textbook problems of finding moment of inertia. Please note that finding moment of inertia is not difficult, but it takes practice! In the most complicated cases we use specialized software to numerically compute moment of inertia.
Below you can find three examples on how to calculate moment of inertia for a cube and a cylinder. Make sure to understand these examples before you move on to more complex problems (objects). For anything more detailed than this, consult some book on calculus on how to deal with double and triple integrals.
Moment of inertia in general case
The idea is simple - imagine an object is composed of many many infinitesimally small particles of mass $m_i$ at distance $r_i$ from the axis of rotation. To find moment of inertia you just sum distance squared times mass of each particle
$$I = \sum_{i} r_i^2 m_i$$
The above equation works well in discrete case, but in continuous case the sum becomes integral
$$\boxed{I = \iiint_\mathcal{V} r^2 dm} \tag 1$$
where $\mathcal{V}$ is the volume that object occupies in 3D. Now the only thing you have to do is to represent the infinitesimally small mass $dm$ as a function of distance $r$. For objects whose density $\rho$ is constant (homogeneous objects), the mass can be expressed as
$$dm = \rho \cdot dV$$
where $dV$ is infinitesimally small volume. The integral in Eq. (1) now becomes
$$\boxed{I = \rho \iiint_\mathcal{V} r^2 dV} \tag 2$$
Moreover, if object's cross section does not change along the axis of rotation, the above equation can be further simplified:
$$dm = \rho \cdot dV = \rho \cdot (h \cdot dA) = \sigma \cdot dA$$
where $h$ is object length along axis of rotation, and $\sigma$ is mass-area density of the object. The above simplification converts triple integral (volume) to double integral (area), which is much easier to depict. The integral in Eq. (2) now becomes
$$\boxed{I = \sigma \iint_\mathcal{A} r^2 dA} \tag 3$$
where $\mathcal{A}$ is the area that object occupies in 2D. The double integrals are solved either in Cartesian coordinates or polar coordinates, whichever is more suitable for a specific problem. Except for these two, the triple integrals can also be solved in spherical coordinates, but I will focus here only on double integrals.
Cartesian and polar coordinates
In all cases discussed here we assume that the axis of rotation goes through the origin. You are allowed to place the origin wherever it is most convenient to solve a particular problem (integral)!
In Cartesian coordinates, the infinitesimally small area and distance from origin are defined as
$$dA = dx \cdot dy \qquad \text{and} \qquad r^2 = x^2 + y^2$$
and the Eq. (3) becomes
$$\boxed{I = \sigma \iint_\mathcal{A} (x^2 + y^2) dx dy} \tag 4$$
In polar coordinates, the infinitesimally small area is defined as
$$dA = (r+dr)^2 \frac{d\theta}{2} - r^2 \frac{d\theta}{2} = (2rdr + (dr)^2) \frac{d\theta}{2} \approx r dr d\theta$$
where $(dr)^2$ is neglected being much (much!) smaller than $2rdr$. With this Eq. (3) becomes
$$\boxed{I = \sigma \iint_\mathcal{A} r^2 \cdot r dr d\theta} \tag 5$$
It must be noted that Eq. (4) and Eq. (5) are only valid if:
In most textbook problems the above two requirements are satisfied. If not, you have to use Eq. (2) for homogeneous objects or Eq. (1) for non-homogeneous objects.
Example 1: Cube with all sides equal to $L$
Assume that the axis of rotation goes through center of mass and is perpendicular to two faces. Moment of inertia is
$$I = \sigma \int_{-L/2}^{L/2} \int_{-L/2}^{L/2} (x^2 + y^2) dx dy = \frac{1}{6} M L^2$$
where $M = \sigma L^2$ is the total mass of the cube. Integration is done in Cartesian coordinates, and the axis of rotation is placed at the origin.
Example 2: Cylinder with radius $R$
Assume that the axis of rotation goes through center of mass and is perpendicular to cylinder's base. Moment of inertia is:
$$I = \sigma \int_{0}^{R} \int_{0}^{2\pi} r^3 dr d\theta = \frac{1}{2} M R^2$$
where $M = \sigma R^2 \pi$ is the total mass of the cylinder. Integration is done in polar coordinates, and the axis of rotation is placed at the origin.
Parallel-axis theorem
The parallel axis theorem, also known as Huygens-Steiner theorem, relates moment of inertia of an object about axis through center of mass to moment of inertia about arbitrary parallel axis
$$\boxed{I = I_\text{cm} + M d^2} \tag 6$$
where $M$ is total mass of the object and $d$ is distance between the two parallel axis.
Example 3: Cube rotating about edge
In Example 1 we have found moment of inertia for cube that rotates about axis through center of mass. When axis goes along cube edge, moment of inertia is
$$I = \sigma \int_{0}^{L} \int_{0}^{L} (x^2 + y^2) dx dy = \frac{2}{3} M L^2$$
We could have calculated this using the parallel-axis theorem. The axis of rotation is at the distance $d = \frac{\sqrt{2}}{2} L$ from axis that goes through center of mass, hence
$$I = \frac{1}{6} M L^2 + M \bigl( \frac{\sqrt{2}}{2} L \bigr)^2 = \frac{2}{3} M L^2$$