Given a function $$f:\mathbb R \rightarrow \mathbb R$$ such that $f'(x_0)$ is continuous in $x_0$, and $$f'(x_0) = 0$$.
Can one state that $x_0$ is either an extremum (local min/max), or an inflection point (i.e. where f changes concavity upward/downward or downward/upward).
If not, Is there a counter example?
I have this example but here $f'$ is not continuous at $0$:
$f(x) = x^2sin(\frac{1}{x})$ if $x \neq 0$ and $f(0) = 0$
Thanks
Best Answer
Here's a proof of my statement in comment.
Theorem. Let $f(x)$ a function defined on some neighborhood of $x_0$, differentiable in $x_0$ up to order $n$. Suppose that $f^{(k)}(x_0) = 0$ for $k=1,2,\dots,n-1$ and that $f^{(n)}(x_0) \neq 0$. Then $x_0$ is a local extremum if and only if $n$ is even.
Note that we do not require continuity (not even the existence) of the $n$-th derivative in a neighborhood of $x_0$.
We use the notation $f^{(k)}(x)$ to refer to the $k$-th derivative of $f$ in $x$, and $f^{(0)}(x) = f(x)$.
WLOG, assume $x_0=0$ and $f(0) = 0$. Suppose also $f^{(n)}(0) >0$.
By definition we have $$\lim_{x\to 0} \frac{f^{(n-1)}(x)}x = f^{(n)}(0)>0.$$ So there must be a neighborhood $\mathcal N$ of $0$ such that $$\frac{f^{(n-1)}(x)}{x}>0\tag{1}\label{eq1}$$ for all $x\in \mathcal N\setminus \{0\}$. If $n=1$ we are done, since \eqref{eq1} guarantees that $0$ is not a local extremum.
If $n>1$, observe that \eqref{eq1} implies that $$f^{(n-2)}(x)>0\tag{2}\label{eq2}$$ for all $x\in \mathcal N\setminus\{0\}$. In fact, suppose by contradiction there is a $\overline x$ such that $f^{(n-2)}(\overline x)\leq 0$. Then, by the Mean Value Theorem we would have $$\frac{f^{(n-2)}(\overline x)}{\overline x} = f^{(n-1)}(\eta)$$ for some $\eta$ in $\mathcal N$. If $\overline x < 0$, then $\overline x < \eta <0$, and $f^{(n-1)}(\eta) \geq 0$. If $\overline x>0$, then $0< \eta <\overline x$ and $f^{(n-1)}(\eta)\leq 0$. Both situations contradict \eqref{eq1}.
If $n=2$ we are again done since \eqref{eq2} implies $0$ is a local minimum. (This depends on the choice we have done to set $f^{(n)}(0)>0$.)
If $n>2$ we can repeat the previous steps to show that $$\frac{f^{(n-2k-1)}(x)}{x}>0, \ \ \mbox{for}\ \ k=0,1,2,\dots,\left\lfloor\frac{n}2\right\rfloor-1$$ and that $$f^{(n-2k)}(x)>0, \ \ \mbox{for}\ \ k=0,1,2,\dots ,\left\lfloor\frac{n}2\right\rfloor,$$ for all $x$ in $\mathcal N\setminus \{0\}$, which imply our thesis.
You have already been given counterexamples of your assertion in the answer and comments. Here are some other examples to further work on your problem, and some remarks.
$$f(x) = \begin{cases} \frac{ax^n}{n!} + x^{2n}\cos\left(\frac1{x}\right) & (x\neq 0)\\ 0 & (x=0) \end{cases}$$ is an example that satisfies the hypotheses, with $f^{(k)}(0) = 0$ for $k=1,2\dots,n-1$ and $f^{(n)} = a$.
One last non null derivative is required in the hypotheses, as pointed out in comment. Otherwise no conclusion can be drawn. Consider for example
$$f(x) = \begin{cases} x^{2(n-1)}\left[2 + \cos\left(\frac1{x}\right)\right] & (x\neq 0)\\ 0 & (x=0) \end{cases} $$for all $n>1$ has a local minimum in $0$. The derivatives in $0$ are all null up to order $n-1$. The $n$-th derivative does not exist in $0$. As an exercise you can show that $f(x)$, despite the minimum in $0$ is not monotonically increasing (decreasing) in any right (left) neighborhood of $0$.
Similarly $$f(x) =\begin{cases} x^{2n-1}\left[2+\cos\left(\frac1{x}\right)\right] & (x\neq 0)\\ 0 & (x=0) \end{cases}$$ for all $n>1$ does not have a local extremum in $0$. The derivatives behave like those of the function in the previous example.
Finally, even when all derivatives in $0$ are null, nothing can be stated. Consider, e.g.
$$f(x) = \begin{cases} e^{-\frac1{x^2}} & (x\neq 0)\\ 0 & (x=0) \end{cases}$$ and $$f(x) = \begin{cases} \frac{x}{|x|}e^{-\frac1{x^2}} & (x\neq 0)\\ 0 & (x=0). \end{cases}$$