First Derivative Test:
Find all points where $f'(x)=0$ or is undefined, these are called critical points. Any of these could be an extrema. However that's not the only option. The point could also be an "inflection point".
An inflection point $(a, f(a))$, is a point where the derivative $f'(a)=0$, but where $f'(a-\delta)$ and $f'(a+\delta)$, where $\delta$ is a very small number, have the same sign (both $f'(a-\delta)$ & $f'(a+\delta)$ are positive, or both are negative). An inflection point, is neither a maximum or a minimum.
NOTE: This is not the technical definition of an inflection point, but it does provide a method that will work for any continuous function $f: D \subset \Bbb R \to \Bbb R$, assuming the $\delta$ you choose is "small enough".
Second Derivative Test:
Plug in the values you found for $f'(x)=0$ into the second derivative $f''(x)$. If:
$\begin{cases} f''(x)\gt 0 & \text{Then f(x) is a relative minimum} \\
f''(x)\lt 0 & \text{Then f(x) is a relative maximum} \\f''(x)=0 & \text{Then the test is inconclusive} \end{cases}$
So for your example:
$f'(x)=6x^2-4x^3=0$ when $x=0$ or $x=\frac 32$. And $f'(x)$ is a polynomial so it is defined over the entire domain of $f$.
That means that your potential relative maxima and minima are at $(0,f(0))$ and $(\frac 32, f(\frac 32))$.
Let's use the second derivative test:
$f''(x)=12x-12x^2=12x(1-x)=0$ when $x=0$ or $x=1$ and is again defined everywhere. So let's figure out where it's positive and negative by choosing three points $c_1, c_2, c_3$ such that $c_1 \lt 0 \lt c_2 \lt 1 \lt c_3$. How about $c_1=-1$, $c_2=\frac 12$, and $c_3=2$ -- those seem like pretty easy numbers.
Plugging those into $f''(x)$, we get
$f''(-1)=-12-12=-24\lt 0$
$f''(\frac 12)=6-3=3 \gt 0$ and
$f''(2) = 24-48=-24 \lt 0$
So we've found that $f(x)$ is concave up on the interval $(0,1)$ and concave down on the set $(-\infty, 0)\cup(1,\infty)$.
So now let's evaluate the points $x=0$ and $x=\frac 32$ that we found via the first derivative test. We see that $\frac 32 \in (-\infty, 0)\cup(1,\infty)$, so we know that the point $(\frac 32, f(\frac 32))$ is a relative maximum.
On the other hand $x=0$ is inconclusive with the second derivative test. So let's check if it's an inflection point via the method I described above. Let's make our $\delta=\frac 14$ and evaluate $f'(-\frac 14)$ and $f'(\frac 14)$.
$f'(-\frac 14)=6(\frac {1}{16})-4(\frac{-1}{64})=\frac {7}{16} \gt 0$ and
$f'(\frac 14)=6(\frac {1}{16})-4(\frac{1}{64})=\frac {5}{16} \gt 0$
So $(0,f(0))$ is an inflection point of $f(x)$ and thus not an extrema.
Best Answer
As I said in a comment, your calculations are wrong. $f'(0.2)$ is indeed positive, but $f'(0.3)$ is negative.
The first derivative test (checking how $f'$ changes sign at a critical point) and the second derivative test (checking the sign of $f''$ at a critical point) will never give contradictory conclusions, as you thought they did here. (You thought the first derivative was indicating a non-extremum whereas the second derivative was indicating a maximum. But your evidence was wrong, because $f'(x)$ does indeed change sign at the critical point.)
The second derivative test may be inconclusive (this happens precisely when $f''$ vanishes at a critical point), but you will never get contradictory results.