The main challenge in this problem is to predict the solid itself. Take a look at the image below
As we increase the height $z$ of the intersecting plane the section of the solid is a square with a side $a$. Since the solid is created with two cylinders of radius $r$, $a$ and $z$ are related by $$a^2+z^2=r^2$$
The area of the square is $A=a^2$ or in terms of $z$: $$A(z)=a^2=r^2-z^2$$ The method of washers tells us $$V=8\int_0^rA(z)dz=8\int_0^r(r^2-z^2)dz=8\left(r^3-\frac{r^3}{3}\right)=\frac{16}{3}r^3$$
Note that the coefficient $8$ is required since we are considering a $1/8$th part of the solid.
It is difficult for me to produce a diagram for you, but a diagram is critical. Draw the hyperbola $xy=1$ (the part in the first quadrant will be enough). Draw the curve $x=y^{1/2}$. Note that this is the first quadrant part of the familiar parabola $y=x^2$. The two curves meet at the point $(1,1)$.
Now draw the line $y=2$. The region we are rotating is below the line $y=2$, to the right of $xy=1$, and to the left of the parabola $y=x^2$. It is a very roughly triangular region, except that two sides of the "triangle" are curvy.
If you have trouble with the graph, perhaps Wolfram Alpha, or another graphing program, or a graphing calculator, would help, but the diagram is really not difficult.
Now imagine rotating this region about the $y$-axis. We get a solid with a "hole" in it.
A cross-section of the solid at $y$ is a "washer," a circle with a circular hole in it. The outer radius of the washer is given by the "$x$" of the parabola, that is, by $y^{1/2}$. The inner radius is the $x$ of the hyperbola, it is $\frac{1}{y}$. So the area of cross-section at $y$ is
$$\pi\left((y^{1/2})^2-\frac{1}{y^2}\right).$$
For the volume, "add up" (integrate) from $y=1$ to $y=2$. The integration will be very straightforward.
Best Answer
We use the technique of "volume by slicing."
Put the tetrahedron face down on the $x$-$y$ plane. For definiteness, put the point where the mutually perpendicular planes meet at the origin. Put the edge of length $3$ along the positive $x$-axis, and the edge of length $4$ along the positive $y$-axis..
We find the cross-sectional area of a slice parallel to the $x$-$y$ plane, at height $z$.
The cross-section at height $z$ is a triangle similar to the triangle at the bottom, with legs scaled by the factor $\frac{5-z}{5}$.
So the area of cross-section at height $z$ is $\left(\frac{1}{2}\right)(3)(4)\left( \frac{5-z}{5}\right)^2$.
To find the volume, integrate from $z=0$ to $z=5$. So the volume is $$\int_{z=0}^5 \left(\frac{1}{2}\right)(3)(4)\left( \frac{5-z}{5}\right)^2\.dz.$$ Integrate.
Remark: We chose a "calculus" approach because of the tag. There are ways to find the answer using basic geometry. If $3,4,5$ are replaced by $a,b,c$ the volume is $\frac{abc}{6}$.