In my physics class my professor was abusing the derivative, as per so many physics classes I've been in. This time, he took the quantity $(x+dx)(y+dy)$ and argued that the $dxdy$ term should disappear, because it's so much smaller than the rest, (despite $dx, dy$ both being infinitesimal…). In any case, I know this is related to non-standard analysis, or something of the sort, and I was wondering if someone could explain in whatever light is proper, why the product of two infinitesimals can be said to be zero. With whatever wonderfully terrible mathematical rigor that is required.
[Math] Products of Infinitesimals
infinitesimalsnonstandard-analysisphysics
Related Solutions
The problem with your proof is the assertion
Now $dF$ is just the small change in $F$ and $f(x)dx$ represents the infinitesimal area bounded by the curve and the $x$ axis.
That is indeed intuitively clear, and is the essence of the idea behind the fundamental theorem of calculus. It's pretty much what Leibniz said. It may be obvious in retrospect, but it took Leibniz and Newton to realize it (though it was in the mathematical air at the time).
The problem calling that a "proof" is the use of the word "infinitesimal". Just what is an infinitesimal number? Without a formal definition, your proof isn't one.
It took mathematicians several centuries to straighten this out. One way to do that is the long proof with limits of Riemann sums you refer to. Another newer way is to make the idea of an infinitesimal number rigorous enough to justify your argument. That can be done, but it's not easy.
Edit in response to this new part of the question:
Plus we do this sort of thing in physics all the time.
Of course. We do it in mathematics too, because it can be turned into a rigorous argument if necessary. Knowing that, we don't have to write that argument every time, and can rely on our trained intuition. In fact you can safely use that intuition even if you don't personally know or understand how to formalize it.
Variations on your question come up a lot on this site. Here are some related questions and answers.
Best Answer
There are several perfectly rigorous ways to formalize this kind of reasoning, none of which require any nonstandard analysis (which you should be quite suspicious of as it relies on a weak choice principle to even get off the ground).
One of them is, as Robert Israel says, interpreting statements about infinitesimals as statements about limiting behavior as some parameter tends to zero. For example, you can define what it means for a function $f(x)$ to be differentiable at a point: it means there is some real number $f'(x)$ such that (in little-o notation)
$$f(x + \epsilon) = f(x) + f'(x) \epsilon + o(|\epsilon|)$$
as $\epsilon \to 0$. After you prove some basic lemmas about how little-o notation works, you get some very clean and intuitive proofs of basic facts in calculus this way. For example, here's the product rule:
$$\begin{eqnarray*} f(x + \epsilon) g(x + \epsilon) &=& \left( f(x) + f'(x) \epsilon + o(|\epsilon|) \right) \left( g(x) + g'(x) \epsilon + o(|\epsilon|) \right) \\ &=& f(x) g(x) + (f'(x) g(x) + f(x) g'(x)) \epsilon + o(|\epsilon|). \end{eqnarray*}$$
After writing down a bunch of arguments like this, if you're familiar with elementary ring theory it becomes very tempting to think of expressions that are $o(|\epsilon|)$ (meaning they grow more slowly than $|\epsilon|$ as $\epsilon \to 0$) as an ideal that you can quotient out by, and this intuition can also be formalized.
More precisely, in the ring $R = C^{\infty}(\mathbb{R})$ of smooth functions on $\mathbb{R}$, for any $r \in \mathbb{R}$ there's an ideal $(x - r)$ generated by the function $x$, consisting of all functions vanishing at $r$. Working in the quotient ring $R/(x - r)$ amounts to only working with the value at $r$ of a function. Working in the quotient ring $R/(x - r)^2$, though, amounts to working with both the value at $r$ and the first derivative at $r$, with multiplication given by the product rule. Similarly, working in $R/(x - r)^{n+1}$ amounts to working with the value at $r$ and the first $n$ derivatives at $r$.
Taking ideas like this seriously leads to things like formal power series, germs of functions, stalks of sheaves, jet bundles, etc. etc. It is all perfectly rigorous mathematics, and nonstandard analysis is a huge distraction from the real issues.