First Derivative Test:
Find all points where $f'(x)=0$ or is undefined, these are called critical points. Any of these could be an extrema. However that's not the only option. The point could also be an "inflection point".
An inflection point $(a, f(a))$, is a point where the derivative $f'(a)=0$, but where $f'(a-\delta)$ and $f'(a+\delta)$, where $\delta$ is a very small number, have the same sign (both $f'(a-\delta)$ & $f'(a+\delta)$ are positive, or both are negative). An inflection point, is neither a maximum or a minimum.
NOTE: This is not the technical definition of an inflection point, but it does provide a method that will work for any continuous function $f: D \subset \Bbb R \to \Bbb R$, assuming the $\delta$ you choose is "small enough".
Second Derivative Test:
Plug in the values you found for $f'(x)=0$ into the second derivative $f''(x)$. If:
$\begin{cases} f''(x)\gt 0 & \text{Then f(x) is a relative minimum} \\
f''(x)\lt 0 & \text{Then f(x) is a relative maximum} \\f''(x)=0 & \text{Then the test is inconclusive} \end{cases}$
So for your example:
$f'(x)=6x^2-4x^3=0$ when $x=0$ or $x=\frac 32$. And $f'(x)$ is a polynomial so it is defined over the entire domain of $f$.
That means that your potential relative maxima and minima are at $(0,f(0))$ and $(\frac 32, f(\frac 32))$.
Let's use the second derivative test:
$f''(x)=12x-12x^2=12x(1-x)=0$ when $x=0$ or $x=1$ and is again defined everywhere. So let's figure out where it's positive and negative by choosing three points $c_1, c_2, c_3$ such that $c_1 \lt 0 \lt c_2 \lt 1 \lt c_3$. How about $c_1=-1$, $c_2=\frac 12$, and $c_3=2$ -- those seem like pretty easy numbers.
Plugging those into $f''(x)$, we get
$f''(-1)=-12-12=-24\lt 0$
$f''(\frac 12)=6-3=3 \gt 0$ and
$f''(2) = 24-48=-24 \lt 0$
So we've found that $f(x)$ is concave up on the interval $(0,1)$ and concave down on the set $(-\infty, 0)\cup(1,\infty)$.
So now let's evaluate the points $x=0$ and $x=\frac 32$ that we found via the first derivative test. We see that $\frac 32 \in (-\infty, 0)\cup(1,\infty)$, so we know that the point $(\frac 32, f(\frac 32))$ is a relative maximum.
On the other hand $x=0$ is inconclusive with the second derivative test. So let's check if it's an inflection point via the method I described above. Let's make our $\delta=\frac 14$ and evaluate $f'(-\frac 14)$ and $f'(\frac 14)$.
$f'(-\frac 14)=6(\frac {1}{16})-4(\frac{-1}{64})=\frac {7}{16} \gt 0$ and
$f'(\frac 14)=6(\frac {1}{16})-4(\frac{1}{64})=\frac {5}{16} \gt 0$
So $(0,f(0))$ is an inflection point of $f(x)$ and thus not an extrema.
Equations which contains polynomial and trigonometric functions do not show explicit solutions and numerical methods should be used to find the roots.
The simplest root finding method is Newton; starting from a reasonable guess $x_0$, the method will update it according to $$x_{n+1}=x_n-\frac{F(x_n)}{F'(x_n)}$$ So, in your case, $$F(x)=x \cos(x)-\sin(x)$$ I think it is better to let it under this form because of the discontinuities of $\tan(x)$.
You can notice that if $x=a$ is a root, $x=-a$ will be another root. So, let us just focus on $0\leq x \leq 3\pi$. If you plot the function, you notice that, beside the trivial $x=0$, there are two roots located close to $5$ and $8$. These would be the guesses.
Using $F'(x)=-x \sin (x)$, the iterative scheme then write $$x_{n+1}=x_n-\frac{1}{x_n}+\cot (x_n)$$ Let us start with $x_0=5$; the method then produces the following iterates : $4.50419$, $4.49343$, $4.49341$ which is the solution for six significant figures.
I let you doing the work for the other solution.
Best Answer
By the cosine addition formula $f(t)\sin(t)$ is a telescopic sum: $$ f(t)\sin(t) = \frac{\cos(4t)-\cos(14t)}{2}$$ and $f(t)$ is a $2\pi$-periodic, odd function, hence in order to find $\max/\min f$ it is enough to find $\max f$.
Since $$ f'(t) = \frac{5\cos(3t)-3\cos(5t)-15\cos(13t)+13\cos(15t)}{4\sin^2 t} $$ by using Chebyshev polynomials of the first kind we get that $f(t)$ attains its maximum value when $\cos(t)$ is a root of $105 - 3100 x^2 + 27488 x^4 - 103552 x^6 + 187648 x^8 - 161792 x^{10} + 53248 x^{12}$. Numerically $$ \max f \approx 4.4648, \qquad \min f \approx -4.4648. $$ We may notice this is just a bit tighter than the trivial bound $|f|\leq 5$.