I am bad at math and am having an extremely hard time trying to understand the conjugate of a function. I am not sure why the dotted line is the conjugate function. How do I find that? This is my limited understanding of the steps.
1) draw $y^Tx$ which is equals to $x y$ in 2 dimensions. Question: is $x$ my gradient or is $y$ my gradient? Because the original equation is $f(x)$ but now is $f^* (y)$?
2) Maximize the function $x y – f(x)$ with respect to $f$ because I am to find the supremum of the expression. How do I draw this?
So I only know how to draw the line $x y$. Then I am stuck. It would be nice if someone could give a numerical example by stepping through some x or y values ( i am not sure if i am supposed to step through the x or y values). I can only understand after seeing an example with numbers.
Question: Why do I shift the line downwards (blue gap) ? The red gap looks bigger to me and we are finding the maximum.
Best Answer
The dotted line is not the conjugate itself. It is a method to compute $f^*(y)$ for a given $y$. For that given value of $y$ (say $y=3$) you want to solve $\sup_x \{ 3x - f(x) \}$. The supremum is attained where the derivative of $3x - f(x)$ is $0$, i.e., $f'(x) = 3$. By moving a dotted line with slope $3$ up or down until it touches the function, you find such $x$ (a horizontal line would find $x$ such that $f'(x)=0$).