It's simple to see that, for $x>0$, the exponential function is lower bounded by its truncated Taylor expansion:
$$e^x = \sum_{j=0}^\infty \frac{x^j}{j!} $$
If we truncate this at $k< \infty$, we are getting a lower bound, as we are just summing over positive quantities. This gives us:
$$\sum_{j=0}^k \frac{x^j}{j!} < e^x $$
However, in this paper at page 3, after equation 2.4, I have found the following bound. Let $\epsilon \in (0,1]$, and $a \in [-1,1]$
$$e^{\epsilon a/2} \leq 1+ \frac{\epsilon a}{2} + \frac{\epsilon^2}{6}$$
This works numerically, but I don't have any intuition for the reason it works. Let's analyze the last term, which is the most troublesome one. They obviously removed the $a$ term from the "Taylor" expansion. However it's not clear how they played with the $/2$ at the denominator. The right term I expect from Taylor would be
$$\frac{(\epsilon a /2)^2}{2!} = \frac{(\epsilon a)^2}{4*2!} = \frac{(\epsilon a)^2}{8} < \frac{(\epsilon )^2}{8} < \frac{(\epsilon)^2}{6}$$
What this bound is saying is that it bounds the remaining terms of the Taylor expansion centered in 0 of $e^{\epsilon a /2}$
$$\frac{\epsilon}{6} \geq \sum_{k=0}^\infty \frac{1}{k!}(\epsilon a/2)^k $$
From here I don t know how to proceed. I wish I could apply the convergence criteria of the gometric series, but there is $1/k!$ at the denominator that prevents me from doing it, and I don t have other ideas.
Numerically, I also checked if this works also removing the $/2$ at the denominator of the exponent. To my surprise, also the bound they propose don't work anymore.
epsilon = 1
ra=np.linspace(-1, 1, 3000)
one, two, three, four = [], [], [], []
for a in ra:
one.append( np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon)**2/3 ) # original
two.append( np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon*a)**2/3 ) # taylor but with 6
three.append( np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon*a)**2/2 ) # taylor right
four.append( np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon*a)**2/2 ) # taylor right
print("orig \t\t\t", np.all(one))
print("taylor but with 6 \t", np.all(two))
print("taylor right \t\t", np.all(three))
print("taylor but with 4 \t\t", np.all(four))
Can you help me please understand the idea behind this step? Thank you.
Best Answer
It seems that looking at the remainder of the first-degree Taylor polynomial is not good enough to get your inequality. Looking at the second-degree polynomial seems to be barely enough to get your bound.
The Lagrange form of the remainder term $R(x)$ in $e^x = 1 + x + \frac{x^2}{2} + R(x)$ is $R(x) = \frac{1}{6} e^\xi x^3$ where $\xi$ is between $0$ and $x$.
Plugging in $x=\epsilon a / 2$ yields $e^{\epsilon a / 2} = 1 + \frac{\epsilon a}{2} + \frac{\epsilon^2 a^2}{8} + \frac{1}{48} e^{\xi} \epsilon^3 a^3$ for some $\xi$ between $0$ and $\epsilon a/2$.
It now suffices to show $$\frac{\epsilon^2 a^2}{8} + \frac{1}{48} e^{\xi} \epsilon^3 a^3 \le \frac{\epsilon^2}{6}.$$ Or equivalently, it suffices to show $$\frac{a^2}{8} + \frac{1}{48} e^{\xi} \epsilon a^3 \le \frac{1}{6}.$$ This holds because $\xi \le \epsilon |a|/2 \le 1/2$ and so the left-hand side can be bounded as $$\frac{a^2}{8} + \frac{1}{48} e^{\xi} \epsilon a^3 \le \frac{1}{8} + \frac{1}{48} e^{1/2}$$ which is just barely smaller than $1/6$.