I'm trying to implement CP-ALS (alternating least squares algorithm for canonical polyadic decomposition) tensor rank decomposition, but I cannot find any references for good guesses for the matrix initialization. For all intents and purposes, third-order-tensors should suffice. At this point, I'm almost thinking of just using unit matrices.
I have searched Google and Youtube and Wikipedia in vain, and have been unable to find either pseudocode, reference implementations, or just walk-through examples with real numbers. Am I looking in the wrong places?
Thanks!
Best Answer
In my very limited experience, initializing an ALS approach to CP or PARAFAC is very heuristic. If you have a 3 mode tensor, you could use a direct trilinear decomposition on the data as a starting guess.
If you have an N-dimensional tensor, the factor matrices can be initialized as, $$A^{(n)} = R \ leading\ left\ singular\ vectors\ of \ X_{(n)} $$ as outlined on page 472 in this excellent review paper.
Another method includes intializing CP randomly several times. If all decompositions have the same fit to the data, local minima may not be a problem. Rasmus Bro's N-way Toolbox allows you to use a combination of all of the above, and use the one with the best fit. You should look into what Sandia National Lab's MATLAB Tensor Toolbox does as well.
3 months after it was originally asked, but hopefully this helps someone.