I will just throw a few buzzwords at you :-)
The mathematically precise concept of "infinitesimal" is called "differential form". If we fix the Euclidean plane $\mathbb{R}^2$ and think of it as a "differentiable manifold", then every point in the plane has a tangential space that is "isomorph to" (or another copy of) $\mathbb{R}^2$. If we further fix a cartesian coordinate system with coordinates x and y, then a differential form is a gadget that assigns for every point p with coordinates $(x_p, y_p)$ to a tangent vector at that point a real number.
If you think of attaching a vector pointing upwards with the length of on, (0, 1), at every point in the plane, then in our example dx would spit out 0 at every point and dy would spit out 1 at every point.
This is the starting point for modern abstract "coordinate free" differential geometry.
From my experience, these concepts are usually not easy to understand for beginners, so don't worry if you don't understand everything on a first reading.
First note that you don't need the concept of a "differential form" to understand your first example:
Take a rectangle with side lengths a and b, then we have a function that gives the area,
$$
f: \mathbb{R}^2 \to \mathbb{R}
$$
$$
f: (a, b) \mapsto ab
$$
If you increase both of the coordinates by h, then this is just a directional derivative, and since $f$ is differentiable we know that
$$
f(a + h, b + h) - f(a, b) = df*(h, h) + o(h^2)
$$
holds. Here "dx" is just short hand notation for both "h" and "we know that f is differentiable and therefore that the remainder of the right hand side is $o(h^2)$, that is for smaller and smaller h the linear approximation gets better and better". The linear approximation is by definition given by applying the differential $df$ of $f$ to the vector $(h, h)$.
The second example is a little bit more complicated, here we'll really need the concept of "differential forms". To be mathematically precise, we'd have to write
$$
ds^2 := dx \otimes dx + dy \otimes dy
$$
That is, the left hand side is defined by the right hand side, and the right hand side consists for a fixed point of elements of the tensor product of the cotangential space $T^*_pM$ with itself.
This means this gadget eats two tangential vectors on any tangent space and spits out a real number, and this operation is bilinear (linear in both input variables). So, if you fix a point on the plane, you get an element of the space
$$
T^*_pM \otimes T^*_pM
$$
which has an algebraic structure that can be used.
If you would like to learn more about this, I'd recommend any textbook on differential geometry or differentiable manifolds.
Since it is supposed to be solved as a 1-dimensional integration problem, most probably Cavalieri's principle is to be used.
- You have already found that $z \in [0,2]$.
- Now, using Cavalieri's principle means, to integrate the area $A(z)$ of the cross sections at height $z$: $\int_0^2 A(z) \; dz$
- Since $z^2-x < y < 2z-x$, $A(z)$ is the difference of the areas of two isosceles right triangles with side lengths $z^2$ and $2z$:
$$ A(z) = \frac 12 \cdot (2z)^2 - \frac 12\cdot (z^2)^2= 2z^2-\frac{z^4}{2}$$
All together
$$V = \int_0^2\left( 2z^2-\frac{z^4}{2}\right)dz=\frac{32}{15}$$
Best Answer
In his book Infinite Powers, Steven Strogatz mentions this exact issue which you are describing. He considers the difference between 'completed infinity' and 'potential infinity'. I will get to your particular issue soon, but first, I will describe a similar problem that helps pinpoint what is going on.
As you know, division by zero is not allowed in mathematics. However, many people share in the false belief that $$ 1/0 = \infty $$ Why is this? A trained mathematician might revile at the above statement, but actually, it is very plausible; wrong, but plausible. Take a look at this: \begin{align} 1/0.1&=10 \\ 1/0.01&=100 \\ 1/0.001&=1000 \\ 1/0.0001&=10000 \\ 1/0.00001&=100000 \\ &\,\,\,\vdots \end{align} As $x$ approaches $0$, $1/x$ approaches infinity. Therefore, $1/0=\infty$. But wait, that's not what we have shown. What we shown is that as $x$ gets closer and closer to $0$, $1/x$ gets larger and larger. There is a crucial distinction between these two statements that is often glossed over in introductory calculus courses. Once you get to the case $1/0$, all kinds of paradoxes emerge, and so it is with good reason that $1/0$ is undefined. However, considering what $1/0$ may or may not be is not an entirely unfruitful exercise. To the contrary, it might reveal to us one of the most important tools in calculus: the limit. Let's take a look at the graph of $y=1/x$:
As you can see, $1/x$ shoots off into the distance as $x$ gets closer and closer to $0$. We can't say what $1/0$ is; what we can say is that as $x$ gets closer and closer to $0$, $1/x$ gets larger and larger. This is formally written as
$$
\lim_{x \to 0}\frac{1}{x}=\infty
$$
But wait, that's not quite right either! $1/x$ only approaches infinity* when $x$ approaches $0$ from the positive end. What if $x$ is a negative number that is getting closer and closer to $0$? Then, $1/x$ approaches $-$infinity. Perhaps we could write that
$$
\lim_{x \to 0}\frac{1}{x}=\pm\infty
$$
but mathematicians like it if limits have a single, definite value. Therefore, both of the above statements are incorrect, and what we should be writing is this:
$$
\lim_{x \to 0^+}\frac{1}{x}=\infty \text{ and } \lim_{x \to 0^-}\frac{1}{x}=-\infty
$$
(The little '$+$' right next to the $0$ indicates that we are considering the case where $x$ is a positive number that is approaching $0$. Likewise, the '$-$' means $x$ is a negative number that is approaching $0$.)
Don't worry if all of the little details don't make complete sense to you. Just try to remember these two key facts:
Now let's compare what we have learnt about limits with what we think we know about infinitesimals. The biggest problem with the concept of an infinitesimal in my mind is that they suggest that there is a 'smallest possible number'. Actually, when we are working with the standard real numbers, there is no such thing. This should be intuitively obvious: however low you go, you can always go lower. You might also be sympathetic to the idea that $$ 1/\text{infinitesimal}=\infty $$ Again, this is problematic, not least because it treats infinity as if it were a number. Therefore, we should be extremely cautious when someone mentions the words 'infinitesimal', or 'infinitely small'. Often when they do, they are using these terms as a mere shorthand for the limits we worked with earlier. For instance, if I write 'as $x$ becomes infinitely small, $1/x$ becomes infinitely large', then this would be rather sloppy, but it would also be generally understood by those well versed in the fundamentals of calculus. (I would warn against using such language though.)
Other times when people mention infinitesimals they are talking about nonstandard analysis, in which the idea of an infinitesimal is formalised. But let's not get sidetracked. As far as I am concerned, 'infinitesimals' do not exist. This should be your view too. While infinitesimals may be intuitively appealing, we should always confirm that our intuitions are in line with reality. Otherwise, we are asking for trouble.
Finally, we get to your question. If I understand correctly, you are asking about $$ \int_a^b f(x) \, dx $$ As you have already correctly pointed out, integrals are based upon splitting up a curve into many small rectangles, each with a certain width. Let's call this width $\Delta x$. We can approximate the area under the graph as $$ \sum_{a+\Delta x}^b f(x) \, \Delta x $$ Do not despair if the above expression looks unfamiliar to you. All it means is that each rectangle has a fixed width $\Delta x$. The length of each rectangle is dependent on the height of the curve at each point—hence why the length is $f(x)$, where $x$ is a variable that goes from $a$ to $b$. And of course, the approximation comes from summing the areas of the rectangles. To visualise this, here is an animation taken from Wikipedia:
As the animation rightly suggests, the approximations become better as $\Delta x$ approaches $0$. This is where $dx$ steps in. You can imagine that if the rectangles have an infinitely small width, then the rectangles get the area exactly right. Historically, $dx$ was indeed used in this way to represent an infinitesimal change in $x$. However, modern standards of rigour have rendered this interpretation of $dx$ largely obsolete. Because of all the paradoxes they can create, infinitesimals are best avoided in formal mathematics, at least within the context of 'standard' calculus. Instead, $dx$ should be seen as part of a shorthand for a limit expression. For example, if $y=f(x)$, then $dy/dx$ is a shorthand for
$$
\lim_{\Delta x \to 0}\frac{\Delta y}{\Delta x}=\lim_{\Delta x \to 0}\frac{f(x+\Delta x)-f(x)}{\Delta x}
$$
In this case of integrals, we can imagine that
$$
\int_a^b f(x) \, dx
$$
represents the sum of infinitely many infinitely small rectangles of width $dx$. But even just writing this makes me cringe. The fact that the number of rectangles gets bigger and bigger does not mean that there are infinitely many rectangles (hence the difference between 'potential infinity' and 'actual infinity'). And judging by the sentiments you expressed in your question, infinitesimals may be not the right approach for you either. You are absolutely right about the apparent paradox created if we interpret the width of each rectangle as truly 'infinitesimal'. The formal definition of an integral sidesteps this issue entirely by defining $\int_a^b f(x) \, dx$ as the limit of the sum of the areas of the rectangles as $\Delta x$ approaches $0$:
$$
\lim_{\Delta x \to 0}\sum_{a+\Delta x}^b f(x) \, \Delta x
$$
A pattern seems to have emerged. Every time you catch yourself thinking about infinitesimals, think abouts limits! You don't need to throw your intuition out of the window—if you find infinitesimals useful for building up your mental picture of calculus, then it would be unwise for you to do away with them. Equally though, never forget what it is really going on.
*Be careful with the phrase 'approaches infinity'. This has a very different meaning to 'approaching $5$', say. If I say $x$ is approaching infinity, then all I mean is that $x$ is getting bigger and bigger. That's it.
**In this particular case, the limit 'does not exist', as we get end up with two different answers: $+\infty$ and $-\infty$, depending on whether we approach $0$ from 'above' or 'below'. Therefore, we have to restrict ourselves to considering the case where $x$ approaches $0$ solely from the positive end, or solely from the negative end.