Minimize $f(x) = x_1^2 -x_1x_2+2x_2^2-2x_1+e^{x_1+x_2}$
a) Write down the necessary conditions of first order. Are they
sufficient? Why?b) Is the point $(0,0)$ optimal?
c) Find a direction $d\in\mathbb{R}^2$ such that $\nabla^t
f((0,0))d<0$d)Minimize the function on $(0,0)$ on the direction found in c)
$$\frac{\partial f}{\partial x_1} = 2x_1-x_2-2+e^{x_1+x_2}$$
$$\frac{\partial f}{\partial x_2} = -x_1+4x_2+e^{x_1+x_2}$$
$$\nabla f(0,0) = [-1 \ 1]\neq 0$$
a) I don't understand this question. There are no sufficient first order conditions, just second order ones. I think it asks me to find all the point that satisfies the first order necessary condition and then verify if they satisfy the second order ones, so:
$$\frac{\partial f}{\partial x_1} = 2x_1-x_2-2+e^{x_1+x_2} =0 \\\frac{\partial f}{\partial x_2} = -x_1+4x_2+e^{x_1+x_2} = 0$$
If I substract the second from the first:
$$3x_1-5x_2-2 =0 \implies 5x_2 = 3-3x_1 \implies x_2 = \frac{3-3x_1}{5}$$
so
$$\frac{\partial f}{\partial x_1} = 2x_1-(\frac{3-3x_1}{5})-2+e^{x_1+\frac{3-3x_1}{5}} = 0 \implies ?$$
It is not trivial to find the zeros of this, I guess. So did I understand the exercise correctly?
b) The point is not optimal
c) If I take $d = -\nabla f(0,0) = [1 \ -1]$ is is a descent direction
d) $$f(-\lambda,\lambda) = \lambda^2 +\lambda^2 +2\lambda^2 +\lambda + e^{0} = 4\lambda^2 + \lambda + 1 = 0\implies \min f(-\lambda,\lambda) = -\frac{b}{2a} = -\frac{1}{8}$$
So what should I do in a) and are my b), c) and d) ok?
Best Answer
Your procedure is fine (except a small typo) and you are stuck for solving $$g(x_1)=\frac{\partial f}{\partial x_1} =\frac{7 }{5}x_1+e^{\frac{8 x_1-2}{5}}-\frac{8}{5}=0\tag 1$$ The solution of it is given in terms of Lambert function
$$x_1=\frac{8}{7}-\frac{5}{8} W\left(\frac{8 }{7}e^{10/7}\right)\approx +0.330449$$ $$x_2=\frac{2}{7}-\frac{3}{8} W\left(\frac{8 }{7}e^{10/7}\right)\approx -0.201730$$ You can approximate the value of $W(.)$ using the expansion given in the linked Wikipedia page.
For these values, you should get $$f=\frac{7}{8} W\left(\frac{8 }{7}e^{10/7}\right)+\frac{7}{16} W\left(\frac{8 }{7}e^{10/7}\right)^2-\frac{8}{7}\approx 0.73372$$ while for $x_1=x_2=0$ you would get $f=1$.
Edit
If you find this exact solution too complex because of the Lambert function, you could notice by inspection that equation $(1)$ has a single root between $0$ and $1$ since $g(0)=\frac{1}{e^{2/5}}-\frac{8}{5}\approx -0.93$ and $g(1)=e^{6/5}-\frac{1}{5}\approx 3.12$. Using Newton method with $x_1^{(0)}=0$ would give a first estimate $x_1^{(1)}\approx 0.376$ (this is an overestimate of the solution by Darboux-Fourier theorem). The corresponding $x_2^{(1)}\approx -0.174$ and for these values $f \approx 0.739$.