I am self-learning basic optimization theory and algorithms from An Introduction to Optimization by Chong and Zak. I would like someone to verify my solution to this problem, on finding the minimizer/maximizer of a function of two variables, or any tips/hint to proceed ahead.
For each value of the scalar $\beta$, find the set of all stationary points of the following two variables $x$ and $y$
$$f(x,y) = x^2 + y^2 + \beta xy + x + 2y$$
Which of those stationary points are local minima? Which are global minima and why? Does this function have a global maximum for some value of $\beta$.
Solution.
We have:
\begin{align*}
f(x,y) &= x^2 + y^2 + \beta xy + x +2y\\
f_x(x,y) &= 2x+\beta y + 1\\
f_y(x,y) &= \beta x + 2y + 2\\
f_{xx}(x,y) &= 2\\
f_{xy}(x,y) &= \beta \\
f_{yy}(x,y) &= 2
\end{align*}
By the first order necessary condition(FONC) for optimality, we know that if $\nabla f(\mathbf{x})=0$, then $\mathbf{x}$ is a critical point.
Thus,
\begin{align*}
f_x(x,y) &= 2x+\beta y + 1 = 0\\
f_y(x,y) &= \beta x + 2y + 2 = 0
\end{align*}
Solving for $x$ and $y$, we find that:
\begin{align*}
x = \frac{\begin{array}{|cc|}
-1 & \beta \\
-2 & 2
\end{array}}{\begin{array}{|cc|}
2 & \beta \\
\beta & 2
\end{array}}=\frac{-2+2\beta}{4-\beta^2}=\frac{2\beta-2}{4 -\beta^2}
\end{align*}
\begin{align*}
y = \frac{\begin{array}{|cc|}
2 & -1 \\
\beta & -2
\end{array}}{\begin{array}{|cc|}
2 & \beta \\
\beta & 2
\end{array}}=\frac{-4+\beta}{4-\beta^2}=\frac{\beta -4}{4 – \beta^2}
\end{align*}
The second order necessary and sufficient conditions for optimality are based on the sign of the quadratic form $Q(\mathbf{h})=\mathbf{h}^T \cdot Hf(\mathbf{a}) \cdot \mathbf{h}$.
The Hessian of $f$ is given by,
$$Hf(\mathbf{x})=\begin{array}{|c c|}
2 & \beta \\
\beta & 2
\end{array}$$
Thus, $d_1 = 2 > 0$ and $d_2 = 4 – \beta^2$. Thus, $f$ has a local minimizer if and only if $4 – \beta^2 > 0$. $g(\beta) = 4 – \beta^2$ is a downward facing parabola. So, the values of this expression positive, if and only if $-2 < \beta < 2$. The function $f$ has no global maximum.
Question. How do I find the actual global minima?
Best Answer
Using elementary algebra, we have
$$f(x,y)=x^2+x(\beta y+1)+2y+y^2$$
Applying well known method,
$$ax^2+bx+c=a(x-m)^2+n$$
$$m=-\frac{b}{2a}, n=-\frac{\Delta}{4a}$$
In this case, we have
$$\begin{align}m:=-\frac{\beta y+1}{2}, ~n:&=\frac 14 \left(y^2(4-\beta ^2) + 2y(4-\beta)-1\right)&\end{align}$$
which gives,
$$f(x,y)=\left(x+\frac{\beta y+1}{2}\right)^2+\frac 14 \left(y^2(4-\beta ^2) + 2y(4-\beta)-1\right)$$
Putting
$$x=-\frac{\beta y+1}{2}$$
We get,
$$f(x,y)≥y^2\left(1-\frac{\beta ^2}{4}\right)+y\left(2-\frac{\beta}{2}\right)-\frac 14$$
Let $\beta ≠±2$, then applying the same method, we have
$$y^2\left(1-\frac{\beta ^2}{4}\right)+y\left(2-\frac{\beta}{2}\right)-\frac 14=\left(1-\frac{\beta^2}{4}\right)\left(y+\frac{4-\beta}{4-\beta ^2}\right)^2+\frac{2 β - 5}{4 - β^2}$$
If $|\beta| >2$, then $\min \left\{f(x,y)\right\}$ doesn't exist.
Let, $|\beta|<2 \iff -2<\beta <2$, then putting
$$y=\frac{\beta-4}{4-\beta ^2}$$
We get,
$$\min \left\{f(x,y)\right\}=\frac{2\beta-5}{4-\beta^2}$$
I'll leave it for you to analyze the case of $\beta=±2.$
This case is equivalent to finding the minimum of the following polynomial:
$$y \left(2-\frac{\beta}{2}\right)- \frac 14.$$
It is possible to see clearly that there is no minimum.