Many times in lectures, I have seen the use of $\mathcal{O}$, especially when writing out Taylor series and so I would like to clarify if my understanding on its use is correct. I came across this question when considering a Taylor series expansion that was written as $$f(x+h)=f(x)+h \cdot f'(x)+\mathcal{O}(h^2)$$ along with the assumption that the second derivative was continuous.
This has prompted me to ask, when $\it{\text{do we}}$ and $\it{\text{can we}}$ we use this notation? My understanding is that the symbol is usually used for series expansions when the higher order terms become 'less significant' as $h \rightarrow 0$. I'm guessing this cannot be freely used and there must be some continuity assumptions applying or something?
As for the above expansion, I would argue that by the continuity of the second derivative, we can use the Lagrange form of the remainder term in Taylor series expansions and thus truncate the infinite expansion to the second order term as follows;
$$f(x+h)=f(x)+h \cdot f'(x)+\frac{h^2}{2!} \cdot f''(\xi)$$ where $\xi$ is in some interval. From this I can see why the $\mathcal{O}$ notation makes sense but I'm not sure if this is the correct explanation behind it.
Best Answer
That is correct. If you want to see it through the mathematical definition of $\mathcal{O}(g(x))$. You can see this:
Well, there is actually a mathematical definition to it. You say that $f(x) \in \mathcal{O}(g(x))$ if you can prove that $\lim_{x \rightarrow 0} \frac{f(x)}{g(x)} = c$ where $c$ is not $\infty$. This is valid in your Taylor series. All terms that include $h^k$ where $k \geq 2$ are in $\mathcal{O}(h^2)$ because $\lim \frac{\alpha h^k}{h^2} = \alpha h^{k-2}$, which is $\alpha$ if $k=2$ and $0$ for $k >2$. Hence, all terms $h^k \in \mathcal{O}(h^2)$, with $k \geq 2$. This means that taylor series will be expressed as $$f(x+h)=f(x)+h \cdot f'(x)+\mathcal{O}(h^2)+\mathcal{O}(h^2)+ \ldots $$ Also, there is another Big-Oh property that uses the fact that $\mathcal{O}(h^2)+\mathcal{O}(h^2) = \mathcal{O}(h^2)$. Hence, all terms $\mathcal{O}(h^2)$ will add up to $\mathcal{O}(h^2)$. So $$f(x+h)=f(x)+h \cdot f'(x)+\underbrace{\mathcal{O}(h^2)+\mathcal{O}(h^2)+ \ldots}_{\mathcal{O}(h^2)} $$