I want to make sure I understood the concept behind SVD for image compression.
So, we start off with a rectangular $m \times n$ matrix that stores all the pixel values of the image. We then compute the SVD of this matrix to get two orthogonal matrices that contain information about the rows and the columns of the original matrix and a diagonal matrix which contains singular values that determine the importance of each rank-$1$ matrix. We then truncate some of the rank-$1$ matrices if their corresponding coefficient in the diagonal matrix is below some threshold value. Say, the number of modes is $k$, the total number of values we need to keep track of will be $k(m + n +1)$.
But once we need to reconstruct the image , we'll have to multiply the three matrices together resulting in a $m \times n$ matrix again.
So, the image is represented as the $3$ matrices in memory but when we want to view the image, only then, does the processor reconstructs the image from the $3$ matrices. Otherwise, the image is just saved in the form of $3$ matrices to save memory.
Best Answer
The SVD decomposes a matrix as a weighted sum of matrices which are themselves outer product of two vectors. Hence you trade $mn$ coefficients for $k(m+n)$, where $k$ is the number of weights retained.
For compression to be effective,
$$k(m+n)\ll mn$$ must hold.