I need a neural network to output a non-singular matrix. However, because of the black-box nature of neural networks I cannot guarantee it. My idea was to take the output of the network (a $n\times n$ matrix) and pass it through a function with above properties.
I need differentiability to perform backpropagation and surjectivity to ensure generality.
Formally: I am searching for a differentiable function
$$f:M_n(\mathbb{R}) \to GL(n, \mathbb{R})$$
with $f(M_n(\mathbb{R}))=GL(n, \mathbb{R})$ where $M_n(\mathbb{R})$ is the set of $n \times n $ matrices over $\mathbb{R}$ and $GL(n, \mathbb{R}) \subset M_n(\mathbb{R})$ denotes the subset of non-singular matrices.
Are you aware of such a function? Alternatively, do you have an alternative idea to solve my problem?
Best Answer
For real-valued matrices, if such a map $\mathbb{R}^{n \times n} \to \mathbb{R}^{n \times n}$ shall be surjective on the non-singular matrices, this does not work:
Consider any two matrices $ A, B \in \mathbb{R}^{n \times n} $, such that $$ det(f(A)) = -1 \qquad \text{and} \qquad det(f(B)) = 1. $$ Since $\mathbb{R}^{n \times n}$ is path connected, there is a continuous curve $ \gamma: [0,1] \to \mathbb{R}^{n \times n} $ with $$ \gamma(0) = A \qquad \text{and} \qquad \gamma(1) = B. $$
Now, since $ f $ is differentiable, $ det \circ f \circ \gamma $ is a continuous function from $ [0,1] $ to $ \mathbb{R} $, with $$ det(f(\gamma(0))) = det(f(A)) = -1 \qquad \text{and} \qquad det(f(\gamma(1))) = det(f(B)) = 1. $$ So by the intermediate value theorem, there is a $ t \in [0,1] $ with $ det(f(\gamma(t))) = 0 $, which means that $ f(\gamma(t)) $ is singular. But this is a contradiction to the image of $ f $ being non-singular.
However, if you allow for smaller image sets, you may for instance use the map $$ A \mapsto A^TA + 1 $$ with $ A^T $ being the transpose of $A$. Then $ A^TA $ is positive and symmetric, so by the spectral theorem it has an orthonormal basis of eigenvectors $(v_j)_{j \in \{1,\ldots,n\}}$ with eigenvalues $ \lambda_j \ge 0 $.
Now for the symmetric matrix $A^TA + 1$, the $v_j$ also form an eigenbasis, but with strictly positive eigenvalues $ \lambda_j + 1 \ge 1 > 0 $. Now, a matrix is singular if and only if its determinant is 0. And for symmetric matrices, the determinant is the product of its eigenvalues. Since $A^TA + 1$ has only nonzero eigenvalues, it cannot have determinant 0, so it is indeed non-singular.
The map is differentiable in $A$, as the entries of $A^TA+1$ are quadratic expressions in the entries of $A$.