For the derivation of a projection on a linear subspace with the use of a minimization problem I'm stuck with the calculation of my dual.
The objective function is: $\frac12||x-z||^2$ and constraint is: $Ax = 0$
A took the lagrangian
$L(\lambda,x) = \frac12||x-z||^2 + \lambda^T Ax$
To find the dual function by calculating the minimum of the lagrangian with respect to x. I do this calculating the gradient of this and equal this to zero:
$x(\lambda) – z + A^T\lambda = 0$ with $()^T$ is the transpose
So $x(\lambda) = z – A^T\lambda$
Fill in my lagrangian ($q(\lambda)$ is my dual function)
$q(\lambda) = \frac12||(z-A^T\lambda)-z||^2 + \lambda^TA(z-A^T\lambda)$
Work out $=\frac12||-A^T\lambda||^2 + \lambda^TAz – \lambda^TAA^T\lambda$
Use positive homogeneity to remove the minus sign within my euclidean norm
$=\frac12||A^T\lambda||^2 + \lambda^TAz – \lambda^TAA^T\lambda$
Now for getting further I should add up the first and the thirth term.
$ =-\frac12||A^T\lambda||^2 + \lambda^TAz$
But I'm not sure my reasoning right, why this is possible
What I think:
$||A^T\lambda||^2 = (A^T\lambda)^2$ because the square cancels the root of the norm.
I can't find the rule somewhere but is $(A^T\lambda)^2= (A^T\lambda)^TQ(A^T\lambda)$ with $Q = I$ (the identity matrix)
$= \lambda^TAA^T\lambda$
Is this reasoning right or wrong?
Best Answer
The first question is to justify the following equality:
\begin{align} &= \dfrac{1}{2}||A^T\lambda||^2+\lambda^TAz-\lambda^TAA^T\lambda\\ &=-\dfrac{1}{2}||A^T\lambda||^2+\lambda^TAz, \end{align}
which amounts to showing: $$||A^T\lambda||^2=\lambda^TAA^T\lambda.$$
This is indeed true. Since by definition $||x||^2=\langle x,x \rangle=x^T\cdot x,$ for all $x$ in any vector space equiped with an inner product $\langle\cdot,\cdot\rangle$. Thus we calculate $$||A^T\lambda||^2=(A^T\lambda)^T(A^T\lambda) = \lambda^TAA^T\lambda.$$
The second question is whether $$(A^T\lambda)^2 = \lambda^TAA^T\lambda.$$ The confusion here might be due to some notational ambiguity. If by the square $(\cdot)^2$ the matrix multiplication square is meant (which would be the conventional notation), then the equality is $\textbf{not}$ true in general.
If, on the other hand, $(\cdot)^2$ represents $\langle\cdot,\cdot\rangle$, then yes, it is true. But one should be wary with using the power notation when different multiplication operations are involved.