I'm not sure if this is a soft question, and whether it may be too broad or, on the contrary, too localized. Well, in Mathematics the concept of "infinitesimal" has been of extreme importance for centuries.
The present mathematical tecnology allows one, according to the context, to formalize this notion in several ways:
- Differential geometry. The classical epsilon-delta formalism of limits in elementary analysis leads to the concept of first-order (or $n$-th order) approximation in Calculus, hence to many standard notions in differential geometry: the differential of a map between smooth manifolds, jets of a map, tangent vectors, differential forms, and Riemannian metrics.
- Algebraic geometry. The lack of a useful notion of convergence of sequences due to the coarseness of Zariski topology prevents us from using epsilon-delta arguments to define "infinitesimals". But then one retains the notion of first-order (or $n$-th order) "approximation" in a more formal way, e.g. by means of universal properties of modules and derivations (Kahler differentials…), and that of "infinitesimal space" e.g. by means of local Artinian $\Bbbk$-algebras, and of "infinitesimal neighbourhood" e.g. by completion of local rings, formal schemes etc. (But after all the algebro geometric perspective is just a high brow way of doing the formal derivative of polynomials, which coincides with the "topological one" once we work over $\mathbb{R}$ or $\mathbb{C}$)
- Synthetic differential geometry. More recently some mathematicians are exploring the realm of synthetic differential geometry, of which I know nothing except that it kind of unifies the perspectives of the previous two approaches and uses relevant amounts of category theory (please correct me if I'm not being correct).
- Non-standard analysis. The concept of infinitesimal element is of course fundamental in non-standard analysis, where the field $\mathbb{R}$ and the use of epsilon-delta arguments is replaced by the introduction of the field ${}^* \mathbb{R}$ of hyperreal numbers, which hosts several hierarchies of infinitesimals (as well as "infinite elements").
- Noncommutative geometry. According to A.Connes (in his book Noncommutative Geometry there's a paragraph on a "Quantized calculus") given an infinite dimensional separable Hilbert space $\mathcal{H}$ and a certain operator $F$ on it, compact operators on $\mathcal{H}$ with characteristic values such that $\mu_n=O(n^{-\alpha})$, $n\to\infty$, can be interpreted as "infinitesimals of order $\alpha$", while the differential of a "complex variable" (read "operator on $\mathcal{H}$") $f$ is just defined to be the commutator $[F,f]$.
At the risk of seeing my question closed as too vague ("not a real question"), it would be my curiosity to know:
Is there a theory that encompasses all the above instances of "infinitesimals" within a unique formal picture? Or, on the contrary, are some of the above notions of infinitesimals inherently specific to their field and embody formalizations of different heuristic notions? [A situation as in the second question occurs with the notion of "infinity": it seems to me there's almost no deep relation between the infinity as in $\lim_{n\to\infty}$ and the infinite cardinals of Cantor]
Best Answer
I don't know of the Connes calculus, but the others (including nonstandard analysis à la Robinson) have been brought under a common framework using models of synthetic differential geometry. However: it is important to point out that the infinitesimals used in algebraic geometry (for jet bundles, etc.) are nilpotent infinitesimals, whereas the infinitesimals used in nonstandard analysis are invertible. So in a sense the answer to the question is, "yes and no", but I'm going to concentrate here on the "yes".
This is explored in Models for Smooth Infinitesimal Analysis by Moerdijk and Reyes, which I recommend. This book can be read as "applied sheaf theory" or "applied Grothendieck topos theory", where the art is to choose a site (small category + covering sieves) judiciously to achieve several aims at once. In many of the models, one takes the underlying category of the site to be something like affine spectra of commutative rings, except one is not dealing with commutative rings exactly, but with richer algebraic structures called $C^\infty$-rings. The formal definition of these is in terms of a Lawvere algebraic theory which allows one to apply not just polynomial operations but more general operations based on $C^\infty$ functions. So the underlying category of the site in these models is the opposite of finitely generated $C^\infty$-rings, which Moerdijk and Reyes call $\mathbb{L}$ (for "locus").
The representing object which gives the locus of invertible infinitesimals is the spectrum of the $C^\infty$-ring given by $C^\infty$ functions $\mathbb{R} - \{0\} \to \mathbb{R}$, modulo the ideal of functions that vanish on some neighborhood of $0$, aka the $C^\infty$ ring of germs at $0$. This is really not much different from nonstandard infinitesimals: usually infinitesimal elements are thought of in some way as germs of functions at infinity, i.e., of functions $\mathbb{R} \to \mathbb{R}$ modulo those which vanish for sufficiently large $x$ (compare here the infinitesimals of Du Bois-Reymond and Hardy). Here Moerdijk and Reyes use $0$ instead of $\infty$. Either way, there are nonarchimedean elements, i.e., nonzero elements less than any $1/n$ in absolute value.
[In nonstandard analysis, one typically refines this idea by considering germs of functions $\mathbb{R} \to \mathbb{R}$ at an "ideal point at infinity", i.e., at a non-principal ultrafilter $U$ on $\mathbb{R}$, or alternatively germs of functions $\mathbb{N} \to \mathbb{R}$ at a non-principal ultrafilter $U$ on $\mathbb{N}$. The more familiar buzzword here is "ultrapower", but see this MO answer by François Dorais, where the implicit message is that an ultrapower along $U$ is really the same as taking a stalk at $U$. (I call $U$ an "ideal point at infinity" because we can think of a non-principal ultrafilter $U$ on $\mathbb{N}$ as a point in the fiber over $\infty$ with respect to the canonical continuous map $\beta(\mathbb{N}) \to \mathbb{N} \cup \{\infty\}$, from the Stone-Cech compactification of $\mathbb{N}$ to the one-point compactification of $\mathbb{N}$.)]
On the other hand, a typical representing object for nilpotent infinitesimals is the spectrum of the $C^\infty$-ring of functions $\mathbb{R} \to \mathbb{R}$ modulo the ideal of squares of elements which vanish at $0$. The "internal hom" represented by this spectrum gives the tangent bundle functor, and other jet bundles can be similarly represented, by using $C^\infty$-rings with different types of nilpotent elements.
The tricky part of all this is to get the right notion of covering sieves, i.e., of sheaves w.r.t. a Grothendieck topology, to achieve disparate aims. One aim would be to embed the usual category of manifolds fully and faithfully in the category of sheaves, so as to preserve "good colimits", such as a manifold $M$ obtained as a colimit along an open covering of $M$. A different aim would be to arrange the topology so that the locus of invertible infinitesimals, as a presheaf on the site category $\mathbb{L}$, is a sheaf w.r.t. the topology. In summary, both aims can be achieved simultaneously so as to accommodate both nilpotent and invertible infinitesimals.