Question:
If $\theta\in[-\frac{5\pi}{12},-\frac\pi3]$ and maximum value of $\frac{\tan(\theta+\frac{2\pi}3)-\tan(\theta+\frac\pi6)+\cos(\theta+\frac\pi6)}{\sqrt3}$ is $\frac ab$ (where $a$ and $b$ are co-primes), find $a-b$
Attempt $1$:
$\frac{\tan\theta-\sqrt3}{1+\sqrt3\tan\theta}-\frac{\tan\theta+\frac1{\sqrt3}}{1-\frac{\tan\theta}{\sqrt3}}+\frac{\sqrt3\cos\theta}2-\frac{\sin\theta}2$
Taking LCM and getting,
$\frac{-4\sec^2\theta}{(1+\sqrt3\tan\theta)(\sqrt3-\tan\theta)}+\frac{\sqrt3\cos\theta}2-\frac{\sin\theta}2$
Not able to move further.
Attempt $2$:
Converting $\tan$ into $\sin$, $\cos$, taking LCM, and getting
$\frac1{\cos(\theta+\frac{2\pi}3)\cos(\theta+\frac{\pi}6)}+\cos(\theta+\frac{\pi}6)$
Not able to proceed next.
Attempt $3$:
$f'(\theta)=\sec^2(\theta+\frac{2\pi}3)-\sec^2(\theta+\frac{\pi}6)-\sin(\theta+\frac{\pi}6)$
Not able to proceed from here.
Best Answer
Expanding on @Claude Leibovici's answer, rewrite as $$A=\cos(x)-2\csc(2x)$$
Now, $$A'=-\sin(x)+4\csc(2x)\cot(2x)=\frac{4\cos(2x)-\sin^2(2x)\sin(x)}{\sin^2(2x)}$$
$$\implies A'=\frac 4{\sin^2(2x)}\cdot \left( {\cos(2x)-\sin^3(x)\cos^2(x)} \right)$$
As $x\in \Big[-\frac{\pi}{4},-\frac\pi6\Big]$, $-\sin^3(x)>0$ and $\cos(2x)\ge0$. Squares are always non-negative, which tells us that $$A'>0 \space\forall \space x \in\Big[-\frac{\pi}{4},-\frac\pi6\Big]$$
Thus, the maximum value occurs at $x=-\frac \pi6$ $$\implies A_{\text {max}}=\frac{11\sqrt3}6$$
This, in your original expression gives the maximum value as $\frac ab=\frac{11}6$ $$\implies a-b=5$$
(Thanks @Claude Leibovici for reminding me to add the ending lines)