An inflection point is where a curve changes from concave to convex or vice versa. There are two types of inflection points: stationary and non-stationary. Stationary means that at this point the slope (thus $f'$) is $0$. These points are also called saddle-points.
Non-stationary inflection points are different. They are where the slope is at maximum, i.e. you have to maximize $f'$ in order to find them. You know from caculus that you need to look at both the $f'$ and $f''$ derivatives to determine whether a $f$ has a maximum. But now, since we look for maxima of $f'$, we have to look at $f''$ and $f'''$.
I will illustrate the point considering the case for local maximum. To do so we first need a proper definition of local maxima.
Let $f$ be defined in a certain neighborhood $I$ of $c$. The point $c$ is said to be a point of local maximum of $f$ if $f(c) \geq f(x) $ for all $x\in I$.
As an example the function $f(x) =-|x|$ has a local maximum at $0$. Note that there is no talk about derivative of a function in the definition of local maximum. However using basic concepts from calculus one can find sufficient conditions to ensure that some point is a local maximum for a given function.
Thus one simple condition is that $f$ is increasing to the left of $c$ and then decreasing to the right of $c$. This will ensure that the values of $f$ on left as well as right of $c$ don't exceed $f(c)$ and $c$ is then a point of local maximum of $f$.
This is the basic idea (which is only sufficient but not necessary) which we use to check if a given point is a point of local maximum or not. Since this is based on concept of increasing / decreasing nature of functions, it is natural to assume that $f$ is differentiable in some neighborhood of $c$. And then by definition of derivative it can be proved that $f'(c) =0$ if $c$ is a point of maxima. Thus to find the point of maxima we only need to consider the points at which derivative vanishes. In what follows we assume that $c$ is a point with $f'(c) =0$.
And if derivative $f'$ is positive to the left of $c$ and negative to the right of $c$ then $f$ is increasing to the left of $c$ and decreasing to the right of $c$ and therefore $c$ is a point of maximum. This is what is usually called the first derivative test.
If we assume that $f$ is twice differentiable at $c$ and $f''(c) <0$ then this automatically ensures that $f'$ is positive to the left of $c$ and negative to the right of $c$ so that $c$ is point of maxima by first derivative test. In most cases computing $f''$ at a point $c$ is easier than verifying the change of sign of $f'$ around $c$. This is called the second derivative test. And you can see that it only ensures that the check mentioned in first derivative test is satisfied. But then it has a limitation that it can fail if $f''(c) =0$.
Based on actual function $f$ one should be able to figure which of the tests requires less effort and usually that particular test is chosen. You should observe that both the tests are sufficient and the first derivative test is more powerful of the two because it requires less conditions on $f$ and it can succeed in cases where second derivative test fails (when $f''(c) =0$).
Best Answer
The extrema are found where the derivative is zero. As zero has no sign, you can't tell a minimum from a maximum.
A minimum is where the slope goes from negative to positive, hence the first derivative is decreasing and conversely a maximum is where the slope goes from positive to negative, hence the first derivative is increasing.
So the sign of the second derivative allows you to tell la minimum from a maximum.
If the second derivative is zero, you need more criteria.