Numerical optimization to find unitary matrix

linear algebralinear-transformationsmatricesnumerical optimizationunitary-matrices

I was trying to use numerical optimization to find a unitary matrix. This unitary matrix is 3×3 and real. It is supposed to transform a group of 3-d real vectors into a more impact representation: 2-d representation. It is possible because those 3-d vectors all lie one the same plane which is not aligned to any plane spanned by basis vectors (i.e. x-y, x-z,y-z) The idea is fairly straightforward:

First, I defined a simple perceptron layer (probably the simplest in the world):
enter image description here
all the activation functions are constant 1. As you all can tell, all the weights can be rearranged can put into a matrix the the output is the result of this weight matrix multiplied with the input 3-d vector:
enter image description here

Then I formulated it into a constrained optimization problem:
enter image description here
the first constraint is to minimize the first entry of the output vector, and the second constraint is to make sure the the norm of the output vector is still 1 (assume all input vector has unit norm).

According to my understanding to linear algebra, the second constraint alone can make sure the weight matrix is unitary because it is supposed to preserve the norm.

However, the result I obtained is rather strange.
The first entry of the output vector is indeed minimized (something like 7.58939e-17 which is good enough for my purpose) and the norm is indeed preserved as 1. I tested 20 vectors and the mean of norm is 1.0 while the variance of the norm is 5.793197272716806e-32. All looks good except the weight matrix is not unitary!
I got it to be

array(

   [[ 3.22204422e-02, -3.82534568e-02,  3.39929299e-18],

   [ 5.25421197e-01,  8.82187722e-01,  2.42364635e-01],

   [-2.15688169e-01, -1.20141415e-01,  9.70185232e-01]])

and it multiplied with its conjugate transpose (or just transpose in this case) is:

array(

   [[ 0.00250148, -0.01681743, -0.00235374],

   [-0.01681743,  1.11306323,  0.01582417],

   [-0.00235374,  0.01582417,  1.00221473]])

which is clearly not identity.
I tried to debug but still have no idea what could possibly went wrong.

Best Answer

Turns out that the constraints I gave in the cost function is not enough to guarantee the matrix to be unitary.

Related Question