As I didn't really get a satisfactory answer (although thanks to user35071 for his link, and to those of you that commented), I have decided to answer my own question as best as I can (if I've missed anything important, please let me know in the comments):
- Calculating closed-form summations from known expressions.
- Approximation of infinite-calculus (forward-difference methods, numerical solutions to PDEs and ODEs, etc.)
It is often the case that we are trying to express the summation of some expression $f(x)$ in closed form, and this can be tricky, using standard-techniques.
For instance, computation of $\sum{x^{2}\:\delta x}$ can be done by using two known formulae:
$$x^{n}=\sum_{k}{\left\{n \atop k\right\}x^{\underline{k}}} \text{ and } \sum{x^{\underline{m}}\:\delta x}=\frac{x^{\underline{m+1}}}{(m+1)}\tag{1}$$
Where $\left\{n\atop k\right\}$ is read "$n$ subset $k$", and is a Stirling number of the second kind. And $x^{\underline{k}}=x(x-1)\cdots(x-k+1)$ is the falling-factorial function (note this is also written as the Pochhammer symbol: $(x)_{k}$).
Expanding $x^{2}$ in terms of falling factorials, we get:
$$x^{2}=x^{\underline{2}}+x^{\underline{1}}\implies\sum{x^{2}\:\delta x}=\sum{x^{\underline{2}}\:\delta x}+\sum{x^{\underline{1}}\:\delta x}$$
Using our known formulae in $(1)$, we get:
$$\sum{x^{2}\:\delta x}=\frac{x^{\underline{3}}}{3}+\frac{x^{\underline{2}}}{2}=\frac{x(x-1)(x-2)}{3}+\frac{x(x-1)}{2}=\frac{x(x-1)(2x-1)}{6}$$
Which corresponds to our standard closed form for $\sum_{k=1}^{n}{k^{2}}$.
If we don't restrict ourselves to integral finite-differences, then we can also use it to numerically approximate derivatives and integrals of continuous functions. There are several types of difference methods often used in numerical approximation to calculus,
$$\Delta_{h}[f](x)=f(x+h)-f(x) \tag{2}$$
$$\nabla_{h}[f](x)=f(x-h)-f(x) \tag{3}$$
$$\delta_{h}[f](x)=f\left(x+\frac{h}{2}\right)-f\left(x-\frac{h}{2}\right)\tag{4}$$
Where $(2)$ is called the forward-difference, $(3)$ is called the backwards-difference, and $(4)$ is called the central-difference. These can be used to approximate the derivative using the following formulae:
$$f'(x)\approx\frac{\Delta_{h}[f](x)}{h}\approx\frac{\nabla_{h}[f](x)}{h}\approx\frac{\delta_{h}[f](x)}{h} \tag{5}$$
Which is often used when a non-analytic evaluation of a derivative is required.
They’re exactly the same, because $\binom{n}m=0$ when $n<m$. This is clear if you define the binomial coefficient $\binom{n}m$ for non-negative integers $m$ and $n$ as the number of $m$-element subsets of an $n$-element set. It’s also easily checked if you define $\binom{x}k$ for real $x$ and non-negative integers $k$ as
$$\binom{x}k=\frac{x^{\underline k}}{k!}=\frac{x(x-1)\ldots(x-k+1)}{k!}\;,$$
where $x^{\underline k}$ is a falling factorial: when $x$ is a non-negative integer less than $d$, one of the factors in the numerator is $0$.
Best Answer
You probably mean $$f(n)=\sum_{k=1}^n \frac{1}{k}\approx \ln n+\gamma $$ As a sum is basically a discrete integral, the finite derivative of a sum over the upper bound gives you back the summand $f'(n)\asymp \frac{1}{n}$. More formal treatment of differentiation of discrete sums is found here.
Anyway, I think you can deal with the finite difference implementation now: the difference between two partial sums for different $n$ shouldn't be a problem.