I have heard that given a function $f$ which has certain relative extrema, if $g$ is monotonic, then $gf$, $g$ composed with $f$, will have the same relative extrema as $f$.
Is this true, and if so, where can I find a proof? I do not have a lot of experience with analysis so if someone could explain this intuitively I would appreciate that a lot. Specifically, I am working with likelihood functions and I think this is the reason why optimizing the log-likelihood function of a sample gives the same maximum likelihood estimator as optimizing the likelihood.
Best Answer
First of all, make sure you understand what is meant: The locations of extrema for $f$ are also the locations of extrema for $g\circ f$. It is not the maximum and minimum values taken on that are the same, but the places where those values occur.
The key here is that $g$ is monotone. Either $g$ is increasing, or $g$ is decreasing. Let me discuss increasing first. That means for every $x \le y$ we have $g(x) \le g(y)$.
Now if $x_0$ is a local maximum of $f$, then for all $x$ sufficiently close to $x_0, f(x) \le f(x_0)$. But since $g$ is increasing, that means $g(f(x)) \le g(f(x_0))$. Since this holds for all $x$ sufficiently close to $x_0$, that means $x_0$ is a local maximum of $g\circ f$ as well.
A local minima $x_1$ works the same way. For all $x$ sufficiently near $x_1, f(x_1) \le f(x)$, so $g(f(x_1)) \le g(f(x))$, so $x_1$ is also local minimum of $g\circ f$.
By dropping the restriction of $x$ to neighborhoods of $x_0$ and $x_1$, the same argument shows that global maximums and minimums of $f$ are also global maximums and minimums of $g\circ f$.
When $g$ is decreasing, the only thing that changes is $g\circ f$ will have minima where $f$ has maxima, and vice versa. That $g$ is decreasing means that if $x \le y$, then $g(x) \ge g(y)$. So if $f(x) \le f(x_0)$, then $g(f(x)) \ge g(f(x_0))$. so if $x_0$ is a (local) maximum of $f$, it is a local minimum of $g\circ f$. And similarly for minimums.