This is what I did, but I'm not sure if this is right.
First I find the second-order partial derivatives, by using the chain rule:
$\frac{\partial u}{\partial \theta}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial \theta}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\Rightarrow\frac{\partial u}{\partial x}\frac{\partial x}{\partial \theta}=\frac{\partial u}{\partial \theta}-\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\Rightarrow\frac{\partial u}{\partial x}=\frac{\partial u}{\partial \theta}\frac{\partial \theta}{\partial x}-\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\frac{\partial \theta}{\partial x}$
$\frac{\partial u}{\partial x}=-\frac{\partial u}{\partial \theta}\cdot\frac{y}{x^2+y^2}+\frac{\partial u}{\partial y}r\cos(\theta)\cdot\frac{y}{x^2+y^2}$
$\frac{\partial u}{\partial r}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial r}\Rightarrow\frac{\partial u}{\partial y}\frac{\partial y}{\partial r}=\frac{\partial u}{\partial r}-\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}\Rightarrow\frac{\partial u}{\partial y}=\frac{\partial u}{\partial r}\frac{\partial r}{\partial y}-\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}\frac{\partial r}{\partial y}$
$\frac{\partial u}{\partial y}=\frac{\partial u}{\partial r}\cdot\frac{y}{(x^2+y^2)^\frac{1}{2}}-\frac{\partial u}{\partial x}\cos(\theta)\cdot\frac{y}{(x^2+y^2)^\frac{1}{2}}$
$\frac{\partial^2 u}{\partial y \partial x}=\frac{\partial}{\partial y}(\frac{\partial u}{\partial x})=-\frac{\partial u}{\partial \theta}\frac{\partial}{\partial y}(\frac{y}{x^2+y^2})+\frac{\partial u}{\partial y}r \cos \theta\frac{\partial}{\partial y}(\frac{y}{x^2+y^2})$
$\frac{\partial^2 u}{\partial y \partial x}=-\frac{\partial u}{\partial \theta}\cdot\frac{x^2-y^2}{(x^2+y^2)^2}+\frac{\partial u}{\partial y}r\cos(\theta)\cdot\frac{x^2-y^2}{(x^2+y^2)^2}$
$\frac{\partial^2 u}{\partial y \partial x}=-\frac{\partial u}{\partial \theta}\cdot\frac{r^2cos(2\theta)}{r^4}+\frac{\partial u}{\partial y}r\cos(\theta)\cdot\frac{r^2cos(2\theta)}{r^4}$
$\frac{\partial^2 u}{\partial y^2}=\frac{\partial u}{\partial r}\frac{\partial}{\partial y}(y\cdot(x^2+y^2)^{-\frac{1}{2}})-\frac{\partial u}{\partial x}\cos(\theta)\frac{\partial}{\partial y}(y\cdot(x^2+y^2)^{-\frac{1}{2}})$
$\frac{\partial^2 u}{\partial y^2}=\frac{\partial u}{\partial r}(\frac{1}{\sqrt{x^2+y^2}}-\frac{3}{2}2y^2(x^2+y^2)^{-\frac{3}{2}})-\frac{\partial u}{\partial x}\cos(\theta)(\frac{1}{\sqrt{x^2+y^2}}-\frac{3}{2}2y^2(x^2+y^2)^{-\frac{3}{2}})$
$\frac{\partial^2 u}{\partial y^2}=\frac{\partial u}{\partial r}(\frac{1}{r}-\frac{3}{2}2r^2\sin^2(\theta)r^\frac{1}{2})-\frac{\partial u}{\partial x}\cos(\theta)(\frac{1}{r}-\frac{3}{2}2r^2\sin^2(\theta)r^\frac{1}{2})$
Now using the fact that $x=r \cos(\theta)$ and $y=r \sin(\theta)$, I find $\frac{\partial u}{\partial x}$ and $\frac{\partial u}{\partial y}$ in terms of $r$ and $\theta$:
$\frac{\partial u}{\partial r}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial r}\Rightarrow\frac{\partial u}{\partial x}\cos (\theta)+\frac{\partial u}{\partial y}\sin(\theta)$
$\frac{\partial u}{\partial \theta}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial \theta}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\Rightarrow\frac{\partial u}{\partial x}\left(-r\right)\sin(\theta)+\frac{\partial u}{\partial y}r \cos(\theta)$
Writing in terms of a matrix:
$\begin{bmatrix}
\frac{\partial u}{\partial r}\\
\frac{\partial u}{\partial \theta}
\end{bmatrix}=\begin{bmatrix}
\cos \theta & \sin \theta\\
-r \sin \theta & r \cos \theta
\end{bmatrix}\begin{bmatrix}
\frac{\partial u}{\partial x}\\
\frac{\partial u}{\partial y}
\end{bmatrix}$
$\begin{bmatrix}
\frac{\partial u}{\partial x}\\
\frac{\partial u}{\partial y}
\end{bmatrix}=\begin{bmatrix}
\cos \theta & \sin \theta\\
-r \sin \theta & r \cos \theta
\end{bmatrix}^{-1}\begin{bmatrix}
\frac{\partial u}{\partial r}\\
\frac{\partial u}{\partial \theta}
\end{bmatrix}$
$\begin{bmatrix}
\frac{\partial u}{\partial x}\\
\frac{\partial u}{\partial y}
\end{bmatrix}=\begin{bmatrix}
\cos \theta & -\frac{\sin \theta}{r}\\
\sin \theta & \frac{\cos \theta}{r}
\end{bmatrix}\begin{bmatrix}
\frac{\partial u}{\partial r}\\
\frac{\partial u}{\partial \theta}
\end{bmatrix}$
From here it's just a simple case of plugging in terms into the expression, which I am too lazy to do. Can anyone confirm that this is indeed correct?
You have correctly found the derivative
$$\frac{dy}{dx}=6x^2-12x-18=6(x^2-2x-3)=6(x-3)(x+1)$$
and where it is zero, but you have not quite got the intervals correct.
The derivatives is positive if and only if
$$(x-3)(x+1)>0$$
which is positive if and only if
$$(x>3 \text{ and } x>-1)\quad \text{ or }\quad (x<3 \text{ and } x<-1)$$
i.e. if and only if
$$(x>3)\quad \text{ or }\quad (x<-1)$$
So the function is (strictly) increasing on the interval $(-\infty, -1]$ and on the interval $[3,\infty)$.
The function is (strictly) decreasing on the interval $[-1,3]$.
Best Answer
Hint: I will show the first derivative of each function below, hope this helps:
for $y=(x-1) (x+1)^3$
you could use the chain rule: $\frac{d}{dx}(u.v)=v.\frac{d}{dx}(u)+u.\frac{d}{dx} (v)$
Choose $$u=x-1, v=(x+1)^3$$
$$\frac{d}{dx}(u)=\frac{d}{dx}(x-1)=1$$ $$\frac{d}{dx}(v)=\frac{d}{dx}(x+1)^3=3(x+1)^{3-1}(x-1)=3(x+1)^2(x-1)$$ $$\frac{d}{dx} (x-1) (x+1)^3 =(x+1)^3+(x-1)(3(x+1)^2)$$
Now for the function: $y=\dfrac{10}{4x^3-9x^2+6x}$ This function is of the form $g=\frac{u}{v}$ and the derivative is:
$$\frac{\frac{d}{dx}(u).v -u\frac{d}{dx}(v)}{v^2}$$ choose $$u=10, v= 4x^3-9x^2+6x$$ $$\frac{d}{dx}(u)=\frac{d}{dx}(10)=0$$ $$\frac{d}{dx}(v)=\frac{d}{dx} (4x^3-9x^2+6x)=12x^2-18x+6$$
$$\frac{d}{dx}(\frac{10}{4x^3-9x^2+6x})=\frac{-10(12x^2-18x+6)}{(4x^3-9x^2+6x)^{2}}$$
The above derivative is not defined where v=0, that is at values:
$$x=0 , x=\frac{9}{8}+i\frac{\sqrt {15}}{8}, \frac{9}{8}-i\frac{\sqrt {15}}{8}$$