I'm a bit confused with how the SVD is used in collaborative filtering. Suppose I have a social graph, and I build an adjacency matrix from the edges, then take an SVD (let's forget about regularization, learning rates, sparsity optimizations, etc), how do I use this SVD to improve my recommendations?
Suppose my social graph corresponded to instagram, and I was tasked with the responsibility of recommending users in the service, based only on the social graph. I would first build an adjacency matrix $\mathbf A$ $(m\times m)$, take the SVD, $\mathbf A = \mathbf{U s V}$, choose the first $k$ eigenvalues, then what?
I would presumably create a new set of matrices:
\begin{align}
\mathbf U_{new} &\sim m\times k \\
\mathbf s_{new} &\sim k\times k \\
\mathbf V_{new} &\sim k\times m
\end{align}
then what does one do?
I've looked on the web, and most links focus on calculating the SVD, but no one tells you what to do with it. So what should I do?
Best Answer
However: With pure vanilla SVD you might have problems recreating the original matrix, let alone predicting values for missing items. The useful rule-of-thumb in this area is calculating average rating per movie, and subtracting this average for each user / movie combination, that is, subtracting movie bias from each user. Then it is recommended you run SVD, and of course, you would have to record these bias values somewhere, in order to recreate ratings, or predict for unknown values. I'd read Simon Funk's post on SVD for recommendations - he invented an incremental SVD approach during Netflix competition.
http://sifter.org/~simon/journal/20061211.html
I guess demeaning matrix A before SVD makes sense, since SVD's close cousin PCA also works in a similar way. In terms of incremental computation, Funk told me that if you do not demean, first gradient direction dominates the rest of the computation. I've seen this firsthand, basically without demeaning things do not work.