I read a book about the singular value decomposition(SVD) and I have a question about it.
I will briefly explain the problem: For a matrix $A$ we have singular vectors $\{v_i,u_i\}$ and singular values $\{\sigma_i\}$ with the properties $$Av_1=\sigma_1u_1,…,Av_r=\sigma_ru_r,Av_{r+1}=0,…,Av_n=0$$
With $r$ as the rank of $A$. So we have $AV=U\Sigma$ with $U$ and $V$ orthogonal and $\Sigma$ as a diagonal matrix.
Now we look at a matrix which changes over time $A(t)$ and we want to observe the derivative of the singular values of $A(t)$.
So $U^TAV=\Sigma$, because $U$ is orthogonal.
$\Rightarrow u^T(t)A(t)v(t)=u^T(t)\sigma (t)u(t)=\sigma(t)$
Now we want to look at the derivative of the left side, which has $3$ terms from the product rule. Now the book says that the first and third are zero because $Av=\sigma u,A^Tu=\sigma v,u^Tu=v^Tv=1$. So the derivatives of $u^Tu$ and $v^Tv$ are zero. So
$$\frac{du^T}{dt}A(t)v(t)=\sigma(t)\frac{du^T}{dt}u(t)=0$$
for the first term. My problem here is that I don't know why this is true. Since $\sigma(t)\neq 0, \frac{du^T}{dt}u(t)$ has to be zero. How do you bring that into the form of $u^Tu$ so that you can use $u^Tu=1$ and the derivative of that is zero.
I'm sorry if this is trivial but I don't really see it. I hope I explained the problem good enough.
Thanks in advance
Best Answer
Just differentiate both sides of $\mathbf{u}^{T}\left(t\right) \mathbf{u} \left(t\right) = 1$ w.r.t. t to get:
$$\frac{d\mathbf{u}^{T}\left(t\right)}{dt} \mathbf{u} \left(t\right) + \mathbf{u}^{T} \left(t\right) \frac{d\mathbf{u}\left(t\right)}{dt} = \frac{d}{dt} \left( 1 \right) = 0$$
Now, recall that $\mathbf{x}^T \mathbf{y} = \mathbf{y}^T \mathbf{x}$ because the inner product is commutative. Hence, the two terms in the LHS of the above equation are equal and are necessarily equal to zero.